1. CloudFormation Intro
Welcome to this section on cloud formation. In this section, we’ll learn how to manage our infrastructure using code. Now, I know that some of you already know about cloud formation. We haven’t covered this in depth yet in the developer course. I know some of you do not know about cloud formation yet, so I just want to outline the fact that it is a very important exam topic. And as such, I’ve included all the basic lectures from the developer course. So you can catch up if you don’t know about it, and if you already know everything about it, you can still watch them. However, you can then proceed to the advanced systems dedicated lecture, which is added later. And basically, they will go over troubleshooting, advanced confirmation options, which are numerous, and Finn it and CFN signal script. I hope you’re ready. Let’s get started.
2. [DVA] CloudFormation Overview
We are getting to the fascinating section on cloud formation, and cloud formation is one of my favourite topics in AWS. It is something I could talk about for hours and hours, but I’ll try to make it short for you and for this exam. So, if infrastructure is good, what is it? Well, currently, we’ve been doing a lot of manual work. I mean, we’ve been doing manual work; we’ve automated it a little bit with elastic beanstalk. We set up our pipelines to be able to automate our CI CD using code bills, et cetera, and code pipelines. But all this manual work that we’ve been doing is kind of tough to reproduce, right? If we want to reproduce it in another region, here we go again. We click, click, and click everywhere. and that’s a bit boring. It would be even more difficult if you wanted to do it in another AWS account. Even if someone in my company went ahead and deleted everything, forcing me to recreate everything in my region, that would be a nightmare.
So what we really want is code for our infrastructure. And so that’s a new concept, and this is quite a new phenomenon, a new train in the IT world called infrastructure as code. And so that means that this code that we’re going to write will be able to be deployed, which will in turn create, update, and delete our infrastructure. This is where CloudFormation comes in. Cloud formation is going to be a declarative way of outlining your infrastructure for any kind of resource. and most of them are supported. For example, let’s say we take a high-level pseudo-cloudformation template and say, “I want a security group and I want two ECU machines using this group,” I want two elastic IPS for these machines, and I want an SJ bucket, and by the way, I want a load balancer that is connected to these machines. So we say it in a declarative way because this is what we want cloud formation to do. And then cloud formation creates all these things for us in the right order with the exact configuration that we specify. And that’s quite nice. So the benefits of cloud formation, which I’m sure you’re already aware of, are that, for starters, we get our infrastructure as code, so no resources will be created manually, which is excellent for control.
All the code can be version controlled, for example, using Git, so we can version control our cloud formation, which is nice. And all the changes to the infrastructure will be reviewed through code review, which is also very nice. In terms of cost, cloud formation itself is free, but each stack that you create has an identifier, so you can easily track the cost of a stack and estimate the cost of your resources using the cloud formation templates themselves. So if you want to have a savings strategy using cloud formation right in your development environment or your small AWS account, you could automate the deletion of all the templates at 5:00 p.m. and then recreate them at 8:00 a.m. safely. And so, because your infrastructure is code, everything will come back up, and you will save a lot of money. Productivity will benefit from cloud formation as well. So you’re able to destroy and recreate your infrastructure on the fly as many times as you want. You can automate the generation of diagrams for your templates, which is quite nice if you create presentations, and it’s declarative programming. So you don’t need to figure out what goes before what in terms of ordering or orchestration; cloud formation tracks and does that for you. There’s a true separation of concerns as well. So you can have as many tags as you want for many apps and many layers. And so it’s quite common to have a VPC confirmation stack that creates all the networks and the subnets. There is an application stack, so for each application you’ll deploy, there’s going to be an application stack. So that’s something we’ve already seen with Elastic Beanstalk.
Each time we created an environment in Elastic Beanstalk, it went ahead and created confirmation templates behind the scenes. And so the idea is that we want to reuse as much work as possible out there, so we’re not going to reinvent the wheel. There are a lot of cloud formation templates on the web that we can already leverage, and we can also leverage the documentation, which is huge and sometimes hard to navigate, but on the documentation you will find everything you will ever need to know. So, cloud formation How do they work? So, we’ll upload the templates to Amazon S3 in the background, and Cloud Formation will pull them from there. And so when we want to update a template, we actually can’t edit a previous template, as we’ll see in the next lecture anyway. But what we have to do is upload a new version of the template to AWS, and then Cloud Formation will figure out what it needs to do to update from version one to version two.
Stacks will be identified by a name, and the names can be very long. And if you delete a stack, every single artefact created by the cloud formation stack is also deleted. So it’s really nice because you can delete all these resources that have been created with one click, and you’re sure that you’re not leaving anything behind. Now, to deploy cloud formation templates, there’s a manual way in which we edit templates in Cloud Formation Designer and use the console to input parameters, and then there’s the automated way, which is to edit templates in a YAML file using a text editor. And then you use the Amazon CLI, or command-line interface, to deploy the templates. It’s the recommended way when you want to take some automation out of your flow, but you’re free to choose either manual or automated. I think they’re both fine. In terms of the building blocks, you’ll learn a lot about them in this section. But there are the template components, and so we’ll get the resources.
And so the resources are basically the same ones that we’ll declare in the template. And that has to be a mandatory section. Okay, your confirmation template cannot work if you don’t specify resources. So machines can have easy access to resources such as elastic, IP, security groups, load balancers, you name it—really, anything you can think of—and parameters. So these are dynamic inputs that you can ask for in your templates. So users will just reference those mappings, which are static inputs for your templates, static variables, and outputs, which is basically saying, “Okay, out of our template, we can export some stuff, and other templates can reference it conditionally,” which is a list of conditions. So if statements basically control what gets created and metadata overall, we’ll see a deep dive into all of those. So don’t worry too much; you’ll get to understand them in their own time. And for templates, you get helpers. And so you can use references, so you can basically link your stuff within your template, and you can use functions to transform data within your templates.
So, again, a high-level overview, but I just wanted to give you this 101. Now this is an introduction to confirmation. To me, it takes over 3 hours to properly learn and master cloud formation, and I do teach it somewhere else. So this section is really meant for you to get a good idea of how it works, but not drill too deep into it because it’s not needed for the exam. So it will be slightly less hands-on than in other sections, but still hands-on enough so you get a good idea of how things work. We’ll learn everything anyway to answer questions for the exam, so do not worry about this. And the exam does not require you to actually write down cloud formation. It will mostly ask you what features to use in cloud formation to perform X, Y, and Z, and you should be fine. So the exam, though, expects you to understand how to read cloud formation. And we’re going to read a lot about cloud formation in this course. So that was a short introduction to cloud formation. In the next lecture, we’ll go ahead with a small example to get an idea of how it actually works. We’ll see you in the next section.
3. [DVA] CloudFormation Create Stack Hands On
Let’s go and create our first CloudFormation template. And this template will create a simple ECU instance in our infrastructure. So we will then go and add an elastic IP to it, and then we’re going to add two security groups to it. but not in this lecture. For now, let’s just forget about the code syntax for cloud formation. We’ll look at the structure of the files later on and understand exactly how they work. So we’ll see how, in no time, we’re able to get started with cloud formation and our EC2 instances. So let’s go to the console. Okay, so let’s get started with cloud formation. So I’m going to go in the direction of cloud formation.
And in here I see three stacks: one for Code Star and two for elastic Beanstalk. So we already had a glimpse of the beanstalk and cloud formation before. So as you can see, this is my production environment stack, and this is my end environment stack. So I’m going to be the proud one. And here in the resources, we can see what was created by Beanstalk. So we can see that 16 resources were created. That includes the Yoro scaling group, the launch configuration, and so on. We’ve already done this once, and we caught a glimpse of cloud formation from the Beanstalk console. But now let’s go to templates and see something really cool. So this is the template, and this is JSON. This is actually horrible to read, but something we can do is click on “View and Designer,” which will take us to the cloud formation designer, where the UI is also atrocious. But here we go.
We can see everything in our cloud formation template and how they relate to one another in this ur. So we can see your scaling policy and auto-scaling group, your launch configuration, your alarm, and your load balancer. And if I look at the load balancer, for example, I can see the configuration of that load balancer in JSON or by converting it into YAML, and of course it is lost. I can see here the load balancer security group, for example. And this is the entire configuration here. Okay, so this is just an introduction to cloud formation. But for now, I’m going to close this designer and return to cloud formation. If you want to do this hands-on with me changing the region, we want to go to US East 1, Northern Virginia. The reason is that all the confirmation templates that we’re going to use in this section are meant to work in North Virginia. They could work somewhere else.
But, just to make sure we’re all on the same page, please select US East 1. Okay, so now let’s go through the creation of our first confirmation stack. So this is great. We have no stacks. We’re going to create a stack and add new resources, which is standard, but we could also use import resources to add existing resources to cloud formation templates. This is out of scope for the exam, so the only way we’re going to do it right now is with the new resources, which is the same as clicking on the Create Stack button in the middle. So now we can use a template, either because it is ready or because we can use a template, several templates, or design one in the designer. But right now, all our templates are ready.
So we’re going to click on “Template is ready,” and then we have two ways of doing it. Either we upload a template file or we go to Amazon Xray and specify an Amazon Xray URL. So in our case, we’re going to upload a template file, choose a file, and choose a file called Zero, just EC Two. And for now, we’re not looking at the code inside that file. And then we’ll click on “Next.” We need to enter a stack name. So I’ll call it, for example, the first stack. And this is just our first stack. And then there are no parameters, so we can just click on Next here. There are no tags, so that’s fine. We don’t have any specified permissions. Stack policy rollback, configuration notification options, and stack creation options are all advanced options out of scope for the exam. So I’m just going to go and click on Next. Okay, so here we go. We have our template URL right here that has been uploaded to Amazon S3, and then there are no parameters and no stack options, so we can just scroll all the way down and click on Create Stack. This will now proceed to create our first confirmation stack for us. And so what we can see is that we are in a different view, under the Events tab. So under the Events tab, what we can see is that right now we are in “Create in Progress.”
So whenever we create a confirmation stack, this is going to be the first state. If I refresh this, you’ll notice two new events, one called “My instance” and one called “Create,” which is currently in progress. And then there was a new event: resource creation was initiated. So that means that my ECTwo instance is actually being created. So if I want to do a sneak peek and go to the EC Two console, now I’m at the EC Two console for the US East one. So this is perfect. Navigate to instances. And I can see an EC2 instance running in here. And if I look at the tags, it’s very cool; we can see that from the tags of that EC2 instance. We are getting some confirmation information, including the logical ID. So my instance, the stack name that it belongs to, and the stack ID, which is the full ARN to my confirmation stack, are here. So this is perfect. My instance has now been created, and if I just refresh, we are now in the create-complete state.
So that means my cloud formation stack has successfully created my EC2 instance. Pretty neat, right? Then we can go to resources, and this shows everything that was created by our confirmation stack. So we have a direct link into the EC2 console for our EC2 instance. It is very handy to have shortcuts like this one. We currently have nothing. A parameter is nothing. The templates that we created and use are referred to as templates. Sorry. So this is a very simple template, and we’ll be able to understand it very, very soon. And then, finally, we can go into the designer to look at our confirmation template and see what it is creating. As we can see, and I’m going to zoom out to find where it is, Here’s my instance. As we can see, we have created an easy instance. Here is the corresponding YAML file, and this is a very simple template in the CloudFormation designer. So that’s it for this introduction lecture to confirmation. We have created our first confirmation stack, and it has created an easy instance for us. And we begin to recognise the power of infrastructure as code. That’s it for me. I will see you at the next lecture.
4. [DVA] CloudFormation Update and Delete Stack
So if we look at the first template we’ve been using, called Zero just EC Two YAML, we’ve been saying it’s a very simple template, and we’ll look at the structure of it in details in the future lectures. But just from reading it, I think it’s quite clear. We are creating resources called myinstance of type EC-2 instance. In terms of properties, we specify the availability zone, the image ID, and thus the AMI ID, which is why we must use UST East because we also specify the availability zone. Finally, T is a two micro instance type. Now what we want to do is update the stack.
And so I’ve created a second stack called EC Two with the security group EIP. So that means that here we’re going to have a lot more complicated templates. I won’t go over it too much, but we’re adding a parameter; we’re changing a little bit. That’s easy. For instance, we’re adding an elastic IP, a security group, and a second security group. So we can expect a lot of updates. So let’s see how we can go about cloud formation and apply this update. So we’ll go in here and then click on Updates. And then you have to choose how you want to update. Do you want to use the current template, replace it, or edit it? I’m going to replace the template, upload a new file, and this time I’m going to upload the file to EC 2 with SGEIP.
So the file is being loaded, it’s being uploaded to Amazon S3, and then I can click on Next. Now I get prompted to enter the value of a parameter. That’s because we have defined a new section in our template. And so we’ll say this is a cool security group, and this is intended to become my security group description. So I click on Next, and then in terms of tag options, we’ll leave the tags, permissions, and so on. As is, we’re not changing anything here. And now we’ll review everything. So the template measures value. So this is great. We scroll down, and at the bottom we have something called a change set preview. This is therefore confirmation from the original templates. Okay, so from this template right here to this new template right here, what is the difference and what needs to change? As we can see here, an elastic IP needs to be added, and the EC2 instance needs to be modified. And there is a replacement true. So that means that the previous EC2 instance will be terminated and a new one will be created. That’s because replacement is true. If replacement was false, that means the EC Two instance will stay in place.
So as we can see, we have three new things: an elastic IP and two security groups. And we have the EC-2 instance being modified and even replaced. So if we’re happy with this change set with these changes in advance, then we can click on “Update Stack” and we’re good to go. So we go back into the events, and as we can see now, the stack is an update in progress for the status, and we need to just wait and see what happens. So I’ll refresh this page, and as you can see, two security groups have been created for me. So the creation is complete. And then, if I keep on refreshing, my EC2 instance will be updated. As we can see, a new physical EC-2 instance was created because replacement was true. So we need to wait for this. So if I go back to my EC2 instance page and refresh, I should see that yes, my second instance is being created right now due to this update. And so the cool thing to see is that the EIP has not yet been created yet.The reason is confirmation; even though we haven’t specified an order in which resources should be created, we know from the stack templates what to create first. So it knew to create the first security groups, then update the EC2 instance, and finally add an elastic IP.
So Confirmation does this for you and determines the best order for your template creation. That is insanely cool. Now I need to wait for everything to be created. So my instance is now created. And now we are getting into the elastic IP creation process. So the creation is in progress, and we should be done very, very soon. As a result, we expect the replacement to be finished as soon as we finish well. So we expect our previous EasyTo instance to be terminated. So let’s just refresh again to see how we are. Yes, the ESP was created, and firsttack is now in full cleanup mode. And during this cleanup, my previous EC2 instance is being deleted. So that means that my previous EC2 instance here should be shutting down and then being terminated. So it’s really cool. This confirmation template did a lot of things for us based on just a few lines of YAML description code. Okay? So if we go into resources now, we can see we have a lot more things. We have a link to the Elastic IP, the EC2 instance, and the two security groups.
And so if we check it out, let’s just click on the EC2 instance right now. So I’m taken directly to the EC2 instance, and I can see that, yes, it has an elastic IP address. So if I click on the elastic IP, I can see it is right here. And if I look at this elastic IP and look at the tags, for example, we can see it is again tagged by cloud formation. So just like my easy two instances, This one is tagged, and we can also take a look at our security groups. So here we have two security groups. So I’m going to go into my security groups and just search for the word stack. Here we go. We have two security groups created for me. So we have our SSH security group, which contains an inbound rule for SSH, and we also have a server security group, which contains a rule for port 80. Okay? And then finally, if we look at the description of this security group, it says this is a cool security group. So that means that the parameter that we somehow defined here and gave a value to was applied to the value of the description of that security group. And we’ll see how that works as well later on in this section. So, very, very cool.
We have created our cloud formation template and are updating it. So now what we can do is look at deletion. So if we go into our cloud formation stack and we’re happy where we are, one way we could do things is go into your instances, right-click here, and then terminate it. But the problem with this is that as soon as you do this, all the other things that were created by confirmation still remain. This includes your security groups as well as your EC to your elastic IP address. So what we can do instead is take this confirmation stack, and then we can click on delete, and by doing so, it will delete everything in our cloud formation stack.
So I’ll just click on “delete stack,” and then, with confirmation, we’ll go into “delete in progress.” And again, if we look at the event list, it knows what to delete first. So it is going to delete the elastic IP first, then it will go ahead and delete the EC2 instance, and finally it will go ahead and delete the security groups. And that’s the whole power of cloud formation. It is very easy to create resources, update them, and delete them without trying to figure out in what order to do things. So as soon as you go and do infrastructure as code, it is extremely important for you to understand that every resource creation, update, and deletion has to happen through cloud formation. And when you get the hang of it, it becomes really, really handy and something you will not be able to go back to. So I’ll just wait for this tap to be silenced, but it should work on my end, and I will see you in the next lecture.
5. [DVA] YAML Crash Course
So YAML is what we use all across AWS for many types, including cloud formation. And so cloud formation supports YAML and Jason’s. And these are basically some scripting languages or data languages that you can use for cloud formation. And to be honest, I’ll be very honest with you: Jason is absolutely horrible for confirmation. It is unreadable. It is unwritten. And YAML is great in so many ways. And this is the language that will be used in the exam to show you some CloudFormation templates. And this is what I’ve been using so far. So let’s learn a little bit about it. Even though we’ve been in touch with it so many times, now is the time, I think, to formalise how YAML works. And so this is a YAML file. As a result, the YAML file contains a large number of key-value pairs, also known as key-value pairs. And to me, they’re very readable objects.
Okay? On the left, we can simply look at it and understand it. If I ask you, what’s the invoice number? You don’t need to be a genius to tell me that the invoice number is 34843. And so this is great. If I tell you the date, you can also tell me the dates. If I ask you, “Hey, what are the quantities of the products we’ve ordered?” You can look at products, and then you can look within the product. There’s a list of quantities, and we have four and one. So you get the idea, right? With YAML, we can have key-value pairs at the top level as well as nested objects. It has support for arrays. So, if we look at product, we can see that there is a minus sign here, which means array. So the product is an array of SKUs, quantities, descriptions, et cetera. And then we can look at, for example, building two. There is a nested object within it called Given Chris Family Demarz. And then your dress also has a nested object within it, which is called line, city, state, and postal. You know, if we look at the lines, we can see that there is multiline string support. So we have a little, like, vertical bar right here. And this is called a multiline string. So we can easily add multiline strings through YAML. And we can also include comments, although they’re not shown on this page.
So YAML is a great way for me to read and write. Obviously, some YAML files can be quite complicated, but overall, we get the idea. Let’s take a look at the simplest of the confirmations we’ve received. We can see this one from the YAML file. The first top-level key was resources. Then there is a second nested object within it called My Instance, which has more nested stuff within it, such as type and properties. And properties were again nested as key-value pairs. If we look at the other one, we can see we have the parameters, top level, the resource, top level, and more Netsys stuff. And if we look at security groups, because there’s a little minus sign right here, it’s a list. So we have a list of security groups that we’ve defined right here. The first security group and the second security group So you can navigate through this and really understand that it looks like we can read this YAML template just using the small concepts we’ve learned before. I encourage you to go online and learn a little bit more about YAML and even practise converting JSON documents to YAML. But overall, I think it’s quite easy to read. And what you really should understand is that you can nest objects separated by the column. You get the key and the value, and using the minus sign, you get to assign a list. So that’s it for YAML. I hope this makes more sense to you now, and I will see you in the next lecture.
