24. GuardDuty – Overview
So guard duty is a very special service that’s kind of hard to understand because we don’t have to do much. But it is an intelligent threat discovery, basically meant to protect your aws accounts. That means that it’s going to run some analysis in the background. You don’t have to do anything. It will use the logs that’s available to it and it will just make sure that it’s protecting you against malicious usage. So it will use a machine learning algorithm, animal detection, and third party data, mingle all this together and tell you what’s up. So to enable it, you just do one click and you get a 30 days trial. You don’t want to install any software, do anything, but then after these 30 days, you start paying.
And it’s not cheap. Now, input data that is going into guard duty includes cloud trail logs. So basically it will look for unusual api calls or unauthorized deployments, that kind of stuff. It will also look at vpc flow logs to detect some unusual Internet traffic or unusual IP addresses that are appearing within your network. It will also look at the dns logs to look for compromise easy to instances that are starting to send encoded data within dns queries. So it’s going to look for a lot of patterns and a lot of things that you don’t want to take care of, that Amazon will take care of for you by analyzing all your network logs, basically, and make sure that if anything is wrong, it’s going to notify you right away.
On top of it, you can integrate this with a twist lambda if you wanted to automate much of this infrastructure. So let’s have a quick look at guard duty in the ui, but there’s nothing much to do. All right, so in the cancel console, just put guard duty in and you’re taken straight to the page. So, as we can see, guard duty is to do intelligent threat detection and protect your address, account and workloads. So it’s continuous, it’s comprehensive, it’s going to allow all these events, including cloud trail and vpc flow logs and dns events. And then it’s going to basically give you some insights. So let’s get started. And it is so easy.
First you need to basically enable guard duty, and it will get some permissions to analyze the cloud trail logs, the vpc flow logs, and the dns logs basically to generate findings. And then you just click on enable guard duty. And that’s basically it. Now it’s going to go overall and do these things and analyze wise all over the time. Now, I don’t recommend that you put this all the time on because it’s going to be expensive. But as you can see on the left hand side, there’s a free trial. And right now I’m on day one of my 30 day trial. So what we get out of it is that we’ll get the number of events processed.Since you enable it.
So you get how many cloudware logs, cloud trail logs, vpc flow logs, dns logs, and that will basically give you an estimated daily cost after the free trial ends. So if you’re very worried about the security of your amazon’s account and you want to make sure you don’t have any people using it for malicious reasons, then this is kind of a nice thing to have. So as you can see, you can set up the settings and see the raw permissions, you can set up some cloud watch events, you can generate simple findings to basically understand the kind of findings that guard duty generates. So let’s have a look in a second. We can also suspend it basically to set all the stuff or disable it so we don’t pay for it anymore.
If we go to list, we can see the trusted IP list and the threat list, basically if you know some IP addresses, et cetera, et cetera. And then for the accounts, basically the accounts that are sharing findings with you. So let’s go back to findings. And as we can see, because we have enabled some sample findings, it will tell us what’s going on. So I say, oh, look at this. This EC two instance is mining bitcoin on your EC two? And so it says, okay, I’ve seen that, it’s mining bitcoin, it’s high severity, you should do something about it. So it’s neat, it’s really, really neat. You get a lot of insights into stuff that can go in on other things, can be access, key ID that can be recognized, there’s some unauthorized access, maybe a brute force on your EC Two instance for rdp.
So someone is trying to get into rdp by brute force, maybe there’s a trojan on your machine. So all these things is what you can get as basically insights and findings and they’re all ranked by severity. So basically the blue is going to be just for informational, so low, then the orange is going to be for medium and the red is going to be for high severity. So that just gives you an idea of all the findings you can get out of it. And it’s kind of nice to see it once, but that’s it. It’s not something that you have to do or nothing much for you to do. Just enable it and then over time it will give you findings and you could set up notifications. So we said you receive emails anytime one of these findings is found.
So for the free trial, I’m going to obviously disable it. And as you can see, it started to already analyze my cottage logs and my vpc flow logs in the meantime. So I’m going to accounts, sorry, settings, and then within it I’m going disable guard duty and save the settings so I don’t get to pay. So disabled it and now I’m done. So that’s it for guard duty. Just remember what it does at a high level, okay? It analyzes your internal accounts and basically try to detect threats or viruses or stuff like this. Stuff that can basically hamper take over your account or try to do malicious stuff with it, such as bitcoin mining. All right, I will see you in the next lecture.
25. GuardDuty – Automations
So one thing I want to show you is how to do automation with guard duty. But I’m pretty sure you already know the answer to this. So you can go to Cloud Watch Event Rules and you create a rule. You’re going to find the service named Guard Duty and then you can look at guard duty findings. And so whenever there’s a guard duty finding, this would create an event. And then maybe a good target for this would be a lambda function that would send this to a slack channel or maybe a sns topic to send yourself an email automatically whenever there is a guard duty finding because usually these are pretty very important to have in real time. So this is the kind of automation we build with guard duty. It’s a very short lecture, I know, but I just wanted to show you it definitely was possible to do so and it’s good to know going into the exam. All right, I hope you like this lecture. I will see you in the next lecture.
26. Macie – Hands On
Hey. So this lecture is going to be a short lecture, but it is about aws or Amazon Messi. And Amazon Messi is here to analyze your data sets in S Three and ensure that your data is your sensitive data is being protected. And though it will help you classify your sensitive, business critical content, for example, if you send credit card data or if you send pii data into S Three, amazon Messi would detect it and we’ll give you some insights and alerts about it. So if you get started with massey, you can see that it is only available right now at this time of recording in two regions only Us. East or us west? So I’ll choose Us East and I’ll click on Enable massey and that will create an im role for massey and Messi will be enabled as is.
So Messi has now been enabled and currently there is nothing to analyze. So we don’t get any insights in our dashboard. So what we’ll do now is that we’ll go ahead and generate some data and we’ll have this into an S Three bucket and masc will automatically analyze that data and we’ll classify it and this will populate this entire dashboard so we have a better idea of how things work. So this aws plug is really great because there is this confirmation template that will create some sensitive data and put it into a three that will be obviously fake data. And so all of this will then happen in the S Three bucket and masc will analyze the data and will show us some insights and create a dashboard and so on.
So if you scroll down in this article, there is a cloud formation template. You can just click here to launch the stack and this will launch the stack in your confirmation template. And you need to make sure that you do this in Us East Northern Virginia. So click on this and make sure you are in the right region. And then you just have the stack name. You don’t need to pass in an S Three bucket name if you want and you don’t need to pass in an S Three bucket name for your lambda. That’s no worries. And you click on Acknowledge and click on Create Stack. Now the stack is getting created, so I’ll just wait until this is over. So my stack has not created.
And as we can see, there was an S Three bucket that was created, a lambda function that is right here and that we can click on. There’s also a Role, there’s a schedule rule. So this is going to be a cloud watch event rule that will trigger that lambda function every 1 minute. Excellence. It actually does not yes, it does trigger the lambda function every 1 minute. This is excellence. And then finally there is an Sqsq that will contain some data in here. So that’s perfect. We have all that ready for the data generated to work. And so if we go into the bucket, as we can see, we get some files already generated. So it’s going to be a zip code, a pgp mix and file name.
So these are all going to be data sets that are going to be faked, and they’re going to be analyzed by massey. So why don’t we go in massey and make sure we can analyze those? So back in massey, I can go to integrations, and then you select your account ID. So this one, I’ll select it, and then you need to integrate S Three resources with massey. So right now, I can only analyze S Three buckets to look for very high sensitive data. So I’ll add my s three buckets and I’ll select the Amazon Messi Activity Generator buckets. So here we go. I’ll click on Add, and now I’ll say, okay, you should analyze everything. It should be full type of classification and so on.
And it would give me some cost estimates and pricing estimates if you could do so. But there’s a 1GB free tier for massey. So right now, everything you would do here is free, and we’ll stop it before it gets into any paid states. So everything looks good. I’ll start the classification. And now the settings have been updated, and everything will be protected by massey. So now we need to wait for Messi to do this job. So I’ll just pause until we’re done. And so I’ve let massey run for about half an hour now, and we are getting some insights into massey. And so, as we can see, 24 of my assets are critical because they’re highly risk level eight, nine, and ten.So we’ll see what that means in a second.
And about 455 events were analyzed, and we have 23 user sessions. So it’s pretty helpful to see this kind of metrics right from the dashboard. And so if we look at here, if we look at the minimum risks being number eight, we get some information around all the kind of events that have happened. And, for example, we can see that there would be some private key, pair, dsca EC private key, encrypted data keywords. We have some rsa private key, some Facebook secrets. github key, slack API tokens. All these things were found within our buckets. And so that’s quite bad. And so obviously, we don’t want to have those in our three buckets. And so this is why masi is here to help us and find that.
So if we go to alerts on the left hand side, we can look at the specific alerts. And here an ssh private key has been uploaded to an Svocet. So this is something that we don’t want to have. So we can click on the event. We look at the description, we get some information, and if we scroll down, we can see in this alert where in which bucket it is, the object key. So we can have a look, and we get some information around the object, ACL the region and the link directly into the S Three objects. And we can look for activity to see if anyone had access to that file and who did what to that file back into the alerts. This is not just for private keys.
This is also for PIA data. So personally identifiable information such as social security numbers. So here we have social security numbers that were uploaded into Amazon S Three. And this is bad. We don’t want to have this ssn being exposed because if they leak, then people would lose their information, and that could be quite bite for them. And so as such, we want to remediate those. So we’ll have a look and say, okay, the address, email name, and national ID were being found. Okay, where is it? It is in this s three bucket. And if you wanted to have a look at the extent of how much it was being accessed, we could again look at the cloud trail activity.
So that gives you a good idea around what massey can do. This is a service that looks a little bit separate from the other services, and there’s lots of customization possible. You can have users and have as many users as you want. You can customize your alerts. You can have some research so you can start looking through some data and look for the kind of events you’re interested into. You have some settings where you can customize the file extensions, the regx, the content type. You can really, really customize a lot of things. For example, for content type, you have all these settings in here, and I’m not going to go over them, okay? But you get the idea that it’s really customizable.
And you can have integrations with multiple accounts. So you can add accounts, and you can also add many buckets to be monitored with massey. So if you are an organization and you need to find out about pi data or leaks or keys being shared into industry buckets and you don’t know how to retrieve those, well, massey will be a great answer to that, and that’s it. So when you’re done, you go to confirmation and you delete this activity generator so it stops generating some data. And so one thing to notice is that S Three buckets right here has been delete, skipped.
So the confirmation templates specified to skip deleting this bucket because it’s a non empty bucket, and otherwise it would have failed. So to make sure that everything is properly deleted and cleaned up, you have to go into S Three and find your massive bucket. So let me find my massive bucket right here that was created by cloud formation, and in here, I’m going to just empty or delete. And this will empty and delete the bucket at the same time. So I’ll just enter the bucket name in here to empty it and delete it. And then it will be a perfect cleanup of our cloud formation. Stack so here we go. I hope you like this lecture on messi. I will see you in the next lecture.
