20. CloudWatch Events – Overview
So now let’s talk about Cloud Watch events. And Cloud Watch Events has been the bread and butter of this course. And really, this is how you bring all these services together in your account and build automations on top of them. So we need to spend a fair bit of time on Amazon Cloud Watch Events. Something I want to bring your attention to is that Amazon has created a new service called Events Bridge. And eventbridge is nothing than Cloudwatch Events being rebranded with a little bit of stuff on it allowing you to have external events from external api providers onto Cloudwatch Events. But that’s about it. Okay? So if you go to eventbridge and create a rule, you can get the exact same outside demo rule.
You can get the exact same options as Cloud Watch Events. For example, the schedule, if you wanted to specify a cron or the event pattern, if you wanted to have an event pattern defined by service, for example, for aws for Cloud Search, and just a random service, right, all Events. And this will give you an Event pattern out of it. So the reason why they have created Amazon Event Bridge is that because I think that Cloudwatch Events at some point will be a spin off on its own being Amazon Event Bridge. Right now the features are somewhat similar. Now you have the option in Event Bridge to have service partners to send data. For example, you can have Data Dog to send data into your Cloud Watch events. And so that’s a bit of expanded features.
But for now, this is about the same feature sets, except from that service partners. So in this course, I’m going to keep on referring to Cloudwatch Events and keep on doing things from Cloudwatch Events because I think this is what the exam will ask you for a long, long time. Okay? But if one day you see Amazon Events Bridge in the Exam, please let me know. And also don’t be worried. This is the very, very same as Cloud Watch Events, okay? So just something I want to call out. Okay, so now for Cloudwatch Events, we have the rules, and we’ve created a few rules so far. And so we’re going to go ahead and create a new rule and see all the options. So something we’ve seen quite a bit now is the schedule. And this allows you to create an event on the schedule.
Either it’s a fixed rate of X minutes, days, or hours, or you can use a chron expression and you can learn about chronic expression online. And this will allow you to be more flexible, for example, saying at 05:00 A. m. Every Monday. This is a way you can specify this using a crown expression. But the idea with these event rules that are scheduled is that you can have some kind of event that will be triggering a target. For example, usually a lambda function, and that lambda function will do something there for every 5 minutes in this case. Okay, so schedule is quite a very common way of creating an event and the other one is obviously the event pattern. So the event pattern allows you to look at aws services and react to the events that happen to them.
So one that’s been quite popular for us has been code pipeline, for example. And we could choose the type of events within the code pipeline that we want to match. So for example, we want to match the pipeline execution, state change and then we’re saying okay, we’re looking for failures or failed states and this creates an event pattern preview that we could edit if we wanted to, if we knew how we want to edit this. So the idea is that every time you select things from the drop down menu here, this creates an event pattern json document and from this usually you understand what this means. So if you look at the source it’s aws code pipeline. The detail type is pipeline, execution, state change and the detail is state to be failed.
So you have full documentation obviously of these event patterns in the areas, documentation for each of the service that has integration with Cloud Watch events. You could also see what the sample events could look like usually when you have any states. And so this shows you the kind of events that get passed on to the right hand side. So the source, while we have so many aws services in there, pretty much all of them I would say, that have some kind of integration with cloud which events. So again, the ones you need to really remember is going to be cloud formation, code star, api, gateway, auto scaling, could build, could commit to deploy and so on.
All the stuff we’re seeing in this course, every time there is some kind of integration with cloudwatch events, I try to let you know but definitely have a look at the list and see and try to imagine all the kind of possible events you could react to and what you could build on top of it. Okay, so then we go into the targets and the targets is what happens when this event on the left hand side happens. And the most popular one is obviously going to be a lambda function because a lambda function allows you to do pretty much anything you want out of it. So a lambda function could be for example, sending some notifications into slack.
And so whatever happens in cloudwatch events triggers a lambda function which sends it into slack. But there are some other really interesting integration you can have. For example, you can launch a batch job, you can send data to Cloud Watch log group, you can start a code build project, you could start a pipeline, you could create some actions on EC Two for example, to create a snapshot to reboot an instance to stop an instance or terminate instances. You could launch an ecs task if you wanted to. You could, for example, send data to a Firehose delivery stream and have maybe firehose send this in return to S Three or elasticsearch or splunk and so on.
You could also have send data into a kinesis stream if you want to have some real time dashboards on top of it, for example, or some real time computations using kinesis analytics, who knows. Then you could send data to an sns topic, to an Sqsq if you wanted to process it. In due time you can launch ssm automations, ssm Upside M you can launch an ssm run command or finally start a step function step machine. So the idea here is that all these things are baked into cloudwatch events because they’re quite common patterns going into the exam. And if you need to extend it to something very, very custom.
Okay, so this is something you need to know going into the exam, especially the most common one kinesis, ssm, sns, sqs step functions and Cloud Watch could build and code pipeline definitely are the ones you need to remember, but pretty much all of them, I would say. So understanding how cloud wash events fits into your automation is absolutely key going into the devops exam to understand how you can build a ton of automations on your aws infrastructure. So we’ve used cloudwatch events in this course pretty extensively and we will keep on using it when we can. So I’m not going to create a role, but you get the general idea of the service. So I hope that’s it for now and I will see you in the next lecture.
21. CloudWatch Events – Integration with CloudTrail API
So we’re back into Cloud Watch events. And let’s say in EC Two, we want to be able to have some kind of notifications anytime someone or something initiates an AMI creation. So this is quite specific and quite a specific need and desire. And if we look at all the events in here, none of these really help us with AMI creation, right? So we know the api call. The api call is called create image. We would like to be able to intercept these api calls into cloudwatch Events. And for this we can use an aws api call via Cloud Trail. So this is a specific event type and it is actually available for every single service in here. So if you choose elasticsearch, then you can use a pi code, blackout Trail and so on. So let’s go back to EC Two and we’ll select api code via Cloud Trail. And so here it is saying that cloud watch Events supports the same read write apis as Cloud Trail does.
And for read only apis, such as those that begin with List Get and Describe, they’re not supported via Cloud which events. But for example, this one named Createimage is definitely not a list Get or describe. So we are able to intercept it. And so we are going to intercept a specific operation and it’s called Create Image. And I’ll do plus. And as you can see up in here, as we can see, if we look at the event now, the source is EC Two, the detail type is aws api call via Cloud Trail, and then the event source is EC Two, and the event name itself is the api call name Create Image, click on Save. And for example, whenever we have this, we can have a target to be an sms topic and to be this one. Who cares, right? But here we are able to track and have an event thanks to the Cloud Trail integration of any api calls made within your account.
That is not a list get or describe. So this is quite powerful because now for every single service in aws and almost every single api call, that is not a list Get or describe, we are able to create an event for them and send this to the targets of our choice, which is really, really cool when you think about it. So this is all thanks to Cloud Trail. So I’ll call this cloud trail api. Intercept in cloud watch events. This is a really long name and I’ll create the rule. And here we go. So now anytime we’ll do a Create Image A, the event will be sent into an sns topic. And this is pretty nice. So remember this going into the exam. Thanks to combining Cloud Watch rules, Cloud Trails, we are able to create some pretty powerful integrations and intercept any api calls. All right, that’s it. I will see you in the next lecture.
22. CloudWatch Events – vs S3 Events
So there is a way to enable and configure event notifications for S three buckets and that is completely unrelated to Cloud Watch events. So let’s go into s three and let’s find and let’s create an Sq sq as well while we’re at it. So we’re going into S three and we’re going to create to use this bucket, for example, AWS DevOps define cloud trail. And what we’d like to do is that any time a cloud trail file is delivered into this bucket, we want to enable some kind of integrations and an S three events, okay? And maybe that s three events will put some data into an Sqs queue. So I’ll name it s three events, sqs. That’s the name of my queue. It’s a standard queue. And I’ll click on Quick create queue.
So it’s been created. Excellent. So now into S three management console, I’m going to go to Properties and I will scroll down, and in there I have Events. And Events allows you to create notifications and saying, okay, whenever there is a put, there’s a new object. So a new object notification. So whenever there is a put and you can have a prefix if you wanted to or a suffix, then you send to either an sns topic or an Sqsq or lambda function. So I’ll choose Sqs and saying send it to this one. So new object notification to Sqs and I save this. So it says, okay, it’s unable to do this because the S three does not allow to publish notifications into the Sqs queue. And so to fix this, we have to go to our Sqs queue.
Let’s go to permissions and edit the policy document. And this is just like an S three bucket policy. This is an sqs policy. And so we need to add in this entire document in here. So let’s go and paste this. So we need to say allow for Sqs send message. So we need to pass in the Sqs queue ARN. So let’s go back to Sqs. Let’s find Sqs in here and let’s find the ARN of the queue. So we have this handy. So here is the ARN, and I will paste this in, okay. And then the source ARN is going to be the bucket name. So the bucket name I am in right now is my AWS DevOps define cloud trail. So I’ll copy this and then I will paste this in here. And this should be it. I’ll review the policy and it’s saying it doesn’t look like a json.
For some reason it doesn’t look like adjacent. So maybe because there’s a space here, let’s try again and everything looks good, we’ll save the changes. And now if we go back to S three and save this, hopefully, yes, it goes through. And now we have one event notification active. So something you can do is add other kind of notifications again for different kind of events. So we have put post, copy, multipart upload, and so on so you can read them all. Okay? And then we can have prefix and suffix, and we can send to an sns topic as well. So we could specify an sns topic, or we could send this to an Sqsq or a lambda function if we wanted to have something extremely custom on top of it. So this works.
And this is one way of creating S Three events. And so we could automate a lot of things thanks to this, so that whenever an object is uploaded or deleted or whatever happens in S Three, then we react to it. But as you can see in here, not all kinds of events are supported by S Three. And so maybe you want to use Cloud Watch event rules for this. So if we go back to Cloud Watch and it’s founded here, here is Cloud Watch, we can use Cloud Watch event rules to catch some events that are not cut by S Three notifications. So they’re very different cloudwatch event roles and S Three notification. And so if you go to the S Three service in here, let me find it, here we go. And we look for object level operations.
If we want to catch those, then we need to make sure that we have a cloud trail that is created for that specific bucket. So let’s look at, say, okay, we want to look at a delete objects api on my bucket by name. And I’ll specify this bucket name in here. So if I wanted Cloud Trail to work, cloudwatch events to work for this, what I would need to do is in Cloud Trail. So let’s go to Cloud Trail. Right now. In Cloud Trail, I would need to either create a trail or modify my current trail. So we could go here and modify this trail. So I’ll configure it to have within history either all the sd buckets in your account selected for api calls, or just the Sri buckets that we want to track by specifying the bucket name and save this.
And now, because Cloud Watch is also looking at the S Three bucket operations, we are able to have cloudwatch event rules for S Three work accordingly. Okay, so this is something you should remember. The S Three events do not give us any bucket level operation. It just gives us some object level operations. And they’re right here, and those are the most important ones, but they’re different from Cloud Watch events, where we can have either bucket level operation or object level operation. But to get object level operations within Cloud Watch events, then we need to make sure that within Cloud Trail, we have enabled the capture of the buckets that we want to track from within Cloud Watch events roles.
And so that is the biggest difference between S Three events and cloudwatch events for S Three is that S Three events are native to the S Three service and they’re a bit more integrated. And if you need something that goes out of this scope though something a bit more custom, then you would need to use Cloud Watch event rules but making sure that you have the events tracked by S three in Cloud Trail in the first place. Okay, so that’s it a bit more complicated but I want let you bring this distinction to your eyes so you really understand this. So hope you like this lecture and I will see you in the next lecture.
