Pass Amazon AWS-SysOps Exam in First Attempt Easily

EC2 for SysOps

1. Section Introduction

Okay, so I am in my AWS Management console and what I'm going to do is to go to the EC2 service and we're going to launch an instance. This is going to be a very simple setup because we want to explore other options later on, but we just need to go ahead and make sure we have the same instance being launched, so when you launch an EC2 instance, make sure you choose Amazon Linux Two AMI so we have the same OS. Do not choose Amazon Linux AMI so Linux Two is selected. I'll use a T-2 micro because it is free tier eligible. I'll configure the instance details, and for now I won't change any of these things because we'll look at some of these options later on, but for now we'll just leave everything as is and click on Add Storage. We'll also have a root device of 8GB of an EBS volume. We'll click on Add Tags for this, and I'll just add a tag and say name my first instance or whatever you want to name it, and then I click It's important to create a new security group, which we'll just call AWS SSH. This will just allow us to SSH into a machine. We can just leave the description blank, and we'll make sure that SSH is enabled on port 22 from any IP address with IPV4. Click on Review and Launch, and then click on Launch. As you can see, you need to select an existing key pair or create a new key pair. Keep pair but you can also click and create a new key pair if you wanted to I acknowledge that key pair because I have it and then I click on Launch instance sour instance is now launching we'll go to view instances and so what we'll have to do is just wait for the instance to be ready and what I want to do is to basically SSH into it just to make sure we are on the same page so I just pause the video my instance has now started as we can see there is an IPV Four public IP that has been assigned so I'll just use that IP to SSH into my instance what I'll do now is go to my terminal if you have Windows remember to use Patty to SSH into your instance otherwise if you have Linux and Mac follow along so as you can see I have my AWS course PM file here so I have to use Ch Mad just to make sure that the permissions are applied correctly to my key and then I'll just SSH EC2 user at my ip minus I to specify the key and here we go I am in my EC2 instance. It says welcome to Amazon Linux too. And I'm able to just see an issue's comments from here. This is all I wanted to do right now, just to make sure that you and I could create an EC2 instance SSH into it. The fun stuff is going to arrive in the next lecture, so see you then.

2. Launching an EC2 Instance

Okay, so in this lecture, we're going to change the EC2 instance type, and that's a common operation required for SysOps. So when you have a T2 micro, for example, we want to upgrade this to a T2 small or a T2 Two Large. You can use whatever instance type you want, but this system will only work with EBS-backed instances. The instance will be terminated. Then, using the settings, we're going to change the instance type. Then we can restart the instance, and we'll make sure that our data has not changed. So let's have a play with this. Okay, so I went to my instance, and what I'm going to do is just hackelo into a hello TXT file. And what they did was create a hello.txt file on my simple instance with the text "hello" in it. So pretty easy. This is just a file I'm creating just to make sure we're not going to lose it. We can check the inside memory as we resize the instance, for example, by doing three M. So as you can see, as far as memory goes, we have 985 megabytes of memory in RAM. And so, basically, I want to run this command after I resize the instance just to make sure that we do indeed get more RAM. Okay, so that's about it. I'm going to log out of my instance and go to my ISTO console. As I mentioned, the first step is to stop the instance. It's because if you right-click and select instance settings, you'll notice that the change instance type option is greyed out and says to do it. The first step is to terminate the instance. So we'll go into instance state and stop it. Click on "Yes." Stop. My instance is now stopping, and we have to wait a little bit until it's fully stopped. Okay, so the instance has now stopped. So I can click on instance settings and click on "Change instance type." Now, from a T to a micro, we can go and upgrade to any of these things. So we can downsize. We can go to a T2 nano, for example, but you can also upsize and go for an M-5 large. Who knows? But for us, just because I don't want to pay too much money, I'm going to just upgrade to a T-2 Small. As you can see, when upgrading, for example, an M-5 large, we have the option of having EBS optimised instances. This is something we'll talk about later on, but as you can see, if we switch to T Two Small, we don't have that option in EBS Optimize, so just something to notice. It's kind of cool. Okay, two small will do it. And now already, the instance type in this console has changed to T Two Small. Now what if I go ahead and start my instance. Yes, I started. So what's going to happen is that, behind the scenes, AWS is going to assign my instance to maybe another piece of hardware. And this is something that's seamless for us because we just upgraded this for the console. But this will happen behind the scenes. So what we're going to get is a new IPV for public IP, obviously, because we started, stopped, and then started our instance. But the EBS volumes that were attached to our instance should remain attached, and so we shouldn't lose any data. So here's our IPV. Four public IPs I will go ahead and just SSH into my instance using this public IP. So I'll modify my command and type in is to user at this IP address. Now I'm back in my instance. What if we use LS to list the files? We still have that hello TXT file, so we haven't lost any data, which is very reassuring, and it says hello, and then if I do free minus M, now we can see that our memory is 1993 megabytes. So we doubled the memory of our instance. And so basically, we just upgraded our instance. This used to be something that was quite complicated to do, but now it's very easy for the console, it's seamless, and we don't lose any data because our instance is EBS packed. So this is something you should know, especially when going into your exam, about how to do these things. But it's quite a simple operation overall. As an exercise, what I'm going to do is just go back to your T-2 micro, because I don't want to pay money and I want to remain within the free tier. So you should do the same. If you have a T-2, go back to two micro one more time to make sure that you are not going to spend money on this. So I hope that was helpful, and I will see you in the next lecture.

3. Changing EC2 Instance Type

So now let's talk about placement groups. Placement groups are a little bit more advanced and we want to use them when we want to have control over how our EC2 instances are going to be placed within the AWS infrastructure. So that strategy can be defined using these places placement groups. So we don't get direct interaction with the hardware of AWS, but we let alias know how we would like our easy to instance to be placed compared to one another. So when you create a placement group you have three strategies available for you. You have the cluster placement group in which your instances will be grouped together in a low latency hardware set up within a single availability zone. This is going to give you high performance but high risk. We'll see this in details in a second. Spread means that your instances are going to be spread across different hardware and there is a restriction on this. That means you can only have seven easy to instance per placement group that's spread per AC. So you would use a spread placement group when you have critical applications. Finally, the last one is a new kind of placement group that is really helpful. It's called partition. It's similar to the spread, meaning that you want to spread your instances but here they're spread across many different partitions and these partitions rely on different sets of racks of hardware within an AZ. What does that mean is that they're still spread but they're not isolated one from another failure. But a partition should be isolated from another partition of failure. The idea with this is that you can scale to hundreds of easy two instances per group and that allows you to run applications such as Hadoop, Cassandra or Kafka. Now let's have a look into each of these placement groups in details. For cluster, that means that all our EC2 instances are on the same rack, which means same hardware and it's in the same availability zone. So as you can see, all these instances are on the same hardware. And so why would you do this? Well, basically, we would place them on the same rack because we want to have a cluster, we want to have super low latency, and we want to have maybe a 10 GB/s network. So that means that we have an amazing network, right? But as a drawback of this great network that we get, we get the chrome that if the rack fails, if there is a failure on the hardware, then all the EC2 instances will fail at the same time. So we have increased our risk to have a failure that's going to be perplexated across our entire stack. So when would we even use this? What's the benefit of having this increased risk? Well, we get great network and so for this that means that we can have a big data job that we'll need to complete very fast. Or maybe we have a requirement to have an application that needs extremely low latency and high network throughput and we're willing to take on the risk to have this failure. So this is something you have to realise it's not for every kind of application but if your application need super high bandwidth and low NC placement groups is kind of a nice the cluster placement group is kind of a nice way of doing it. Now spread is the complete opposite and spread one to minimise the failure risk. And so in this case when we ask for spread placement group all the EC2 instances are going to be located on different hardware. So as you can see here we have three AZ and we have six EC2 and each EC2 instance is on a different hardware. So what does that mean? Well, what we get is that we can span across multiple AZ, and there is a reduced risk of simultaneous failure. Why? Well, because if my hardware one fails I'm pretty sure my hardware two will not fail. And so we've separated the risk of my two instances in the US east one A to fail at the same time. That's the benefit from it. The con is that from this configuration we're limited to seven instances per AC for placement group. So there is a limit to how big your placement group can be and so you need to have an application that's going to be a good size but not too big. The use case would be an application that needs to maximise high availability and reduce the risk and in general for critical applications where your instance failures must be isolated from one another. Remember here we have a limitation of seven instances per AZ per placement group. Now for the partition placement group within the AZ we'll have different partitions. So partitions are a set of racks and so we have here in this example three partitions but we can create up to seven partitions in the partition placement group. So on each partition you will have different EC2 instances. So in this example I have four EC2 instances per partition. And here we can see that within a partition all these easy two instances could fail together. But across two partitions there is no failure that's shared. So this is why you would have partitions and an application that can tolerate a whole right going down as long as your data is as well partitioned. So what can we know about this? Well, there's up to seven partitions per AZ and up to hundreds of easy two instances as part of that placement group and that is the difference versus the spread placement group. The instances in a partition do not share the racks with the instances in other partitions and therefore a partition failure can affect many EC2 but won't affect other partitions. Easy two instances can get access to the metadata representing which partition they belong to. And the use cases for this kind of set up is going to be for distributed big data application usually .So HDFS, HBase, Cassandra and Kafka. All right, so let's have a look in the UI about how we can create placement groups. So let's go ahead with creating our first placement group for this. The menu is on the left hand side under Network and Security you will find placement groups. Create a placement group and the first one is going to be called my High Performance Application. And because this is a high performance application, we want to create, and I just name it this way, I want to use the cluster strategy. Remember, cluster groups, my instances together so that they have high network communication between the two of them. So here we go. I've just created my first placement group, which is a cluster placement group. I can create another one. And this one, for example, will be my critical application. And it's critical because I want to run it as a spread cluster. And here our application is going to be spread out as much as possible. But remember, we can only have seven instances per spread ces per spre. Finally, I'll create one more, my distributed application. This one will be a partition strategy. And I have to select the number of partitions. And the tool tip tells me that the maximum number of partitions is seven. I'll just go ahead and create three. Click on Create, and here we go. We get some information about all of the placement groups we have. But how do we use these placement groups? Well, for this we'll go to instances and launch an instance. We'll create an Amazon Linux two instance and we won't go all the way with creating the instance. Just to show you the option, I'll select a teaching micro, click on Configure Instance Details. And here on the left hand side there is Placement group and I can tick the box and add the instance to a placement group. Here I can create a new placement group or add to an existing one, but we choose one that's existing. So we could select spread. And Spread allows us to launch up to seven more instances into the spaceman group. So there will be seven of them will be spread out across Raz. There will also be partition. And here we can specify the target partition we wanted to one, two or three. Or we could have an order distribution if you wanted AWS to try to equalise the partitioning of our EC2 instances. Or as you can see, cluster is not showing up because Cluster is not available for T two type of instances. So if I select something a little bit more, something like M five, a 24 xlarger, I'm not going to launch it, obviously. So I'll say yes, I would like to continue with this. And I'll say, okay. In this time you're going to launch it as a cluster. And so the cluster placement group right now is available only for instances types that are quite high. So here we go. I could select my cluster type and we'll be done. So that's it. This is how you would assign an easy two instance to a placement group and you can create many easy two instances within a placement group. And then you would click on review and launch. But we're not going to do this because I don't want to pay for an M five, a 24 x for an M five. But at least I hope you understand how placement groups work and how we can use them through console. I hope you like this lecture. I will see you in the next lecture.

4. EC2 Placement Groups

Okay, so let's talk about shutdown behaviour and termination protection. So basically, when we remove our Linux machine or Windows from the OS, how should the instance react? Well, by default it's stopped, but there is a way to have it terminated. And so just remember that this is not applicable when you stop or terminate the console or the API. This is you when you perform the shutdown behaviour from within the OS. And we'll see this in the hands of the CLI attribute for this, by the way, which is called instance-initiated shutdown behavior. which makes sense. Okay, the second thing you need to know is about termination protection, and this is to protect against accidental termination in the absence of a console or CLI. So this is to protect humans against making mistakes or your employees, right? So the exam tip is that if you have an instance with the shutdown behaviour terminate and termination protection enabled, what will happen if we shut down from the instance OS? Well, the instance will still be terminated. So let's have a play with this, just to put it into perspective. So let's go ahead and launch an instance. I'll choose Amazon Linux, two T, two Micro, and then in the instance detail, here we go, we can find the options. So the shutdown behaviour is by default "stop." We can say "terminate." We can also enable termination protection to protect against accidental termination. So we'll enable both options. The rest is going to be the same, and we're going to see what that changes. So we'll click on storage tags, configure Security Group, we'll use an existing one, the alias SSH, we'll review and launch and we'll just launch our instance. We'll say you're using the AWS course keeper, and we'll launch our instance. Here we go. Our instance is now launching. While this launches, I want to show you that you can modify these attributes on the running instance. For example, for this one if I right click and I go to instance setting, I can change the termination protection. So here right now, for the instance I had before, it is disabled, but what I can do is enable it. And now this instance that was launched before cannot be terminated from the console. If I right-click and try to terminate and say "yes," I will terminate. You see, it doesn't work. It's saying these instances have termination protection and will not be terminated. And so I need to disable termination before I'm able to terminate my instance. So it's kind of nice because I've protected it. And for this one, obviously if I try to determine it, it will not allow me to. Okay, so here's a quick example because the second one is running with the shutdown equals terminate and the third one is running with the shutdown equals stop. If we look at it in that light. We can also change this by clicking on Instant Settings and changing shutdown behavior, as we can see this one is set to terminate while the first one that I had before was set to stop, so let's go ahead and SSH into both machines and we'll see what happens, okay. So let me SSH into my second machine on the righthand side of my screen, and this is the instance that has the shutdown behaviour equals terminate my first instance I've shown before is the one right here, and I'll also launch it on the left hand side of my terminal, and it's just a little feature of my terminal; I can have them both side by side just to show you, okay? So, on the left, this instance should stop, and this instance should terminate; what I'm going to do is issue the same command called shut down now, so I'll do sudo shutdown now otherwise it won't work, and now my instance is closing, and here I'll do the exact same thing, pseudo shutdown now, so I stopped both instances the same way, but now if we go to the AWS console, we should see something different. They are shutting down, and I'll just refresh to show you what happened, and now we can see that my second instance, which I just launched with shutdown behaviour equal to terminate, got terminated while my first instance, which was launched before, got stopped, and so this demonstrates basically the setting called shutdown behaviour and how it is impacted. We should shut down now. command okay. So that's about it for demonstrating protection for termination as well as the shutdown behavior, and I will see you in the next lecture bye.

5. EC2 Shutdown Behavior & Termination Protection

Okay, this lecture is super important for you to remember. There's tonnes of exam questions on why you can't launch an EC2. So if you get an instance limit exceeded error. If you get that error, that means that you have reached a number of max EC2 instances you can have in your region. And so this is an account limit. The resolution is to either launch the instance in another region, or you can open a support ticket with AWS to increase your limit of the region. Pretty easy. By the way, the default limit of instances in each region is 20. Before we go into other reasons, let's just go quickly, see how we can troubleshoot this and mimic that error. So when we are into our EC2 console on the left hand side, there is a limit. And here we can see the instance limits, and we can see that in my account, the running on demand EC2 instance has a current limit of 20. And so I could click here to request a limit increase. So let's have a look how we can create 20 instances just for fun and see how it goes. For this, I chose to use the Run Instances CLI command, but you could as well use the console if you wanted to add a command. And the command is going to be right here. The only thing I have to add is the Image ID. So I'll go ahead and get an ImageID from my first instance here's, the AMI ID. And I'll just copy this AMI ID here, which is going to be the Image ID. Okay? So now I'm basically launching 19 T two micros with this AMI ID. That should be it. I click on Enter, and what this will do is that after this is successful, I will have 20 T2 micro instances running in my dashboard. So let's have a look at that work. Here we go. Now, I have a lot of instances that are starting to get started, okay? So that basically makes me reach my limits for the account. Now, if I go ahead and try to launch one more instance of T2 micro using the same AMI ID and click on Enter, this time I get the error occurred instance limit exceeded. And it says the quota allows for zero more running instances. And I requested one. So this is the idea. Now, this is definitely what we wanted to do. So basically the idea is that once I have20 instances running into my account, then things go bad when I start to launch new instances. What I'll do now, right now, is just terminate all these instances, obviously. So let me just right click and I'll click on all of them except the one that's already been terminated and the one I want to keep, and I will go ahead and terminate those instances. Yes, please. Okay, here we go. Now, the other troubleshooting issues you can have is that you may have insufficient capacity error. And so when you get this error, that means that AWS does not have the capacity to launch an instance for you and that's a requested. So it's not your fault, it's not a problem with your account, it's a problem with AWS themselves. And so the resolution is to wait a few minutes before trying again, or if you request maybe five instances, request one at a time, or if it's urgent, maybe you can select a different type of instance and then you will upgrade it later on, just like we saw in this course. So it's just something you should know if it's insufficient instance capacity, it has nothing to do with your account, it has to do with AWS themselves. Finally, and that's the most important, if the instance terminates immediately, so it goes from pending state to terminate right away. So that means that you're able to start it but then something went wrong and it got terminated. And you have to remember these reasons number one is you've reached your EBS volume limit. Number two is your EBS snapshot is correct. Number three is your root EBS volume is encrypted and we don't have the permission to access the Kms key for the decryption or finally the instance stored backed AMI is missing a part. And so these four reasons you need to remember, learn them, just recite them night after night until you know them. And then once you get this, if you ever get this in real life, to find the exact reason you need to check the description tab and there's a state transition reason label which I will show you. Now, I cannot trigger this problem, but you need to remember that if your instance goes from pending to terminated, that means that one of these four reasons happened and they ask about it in the exam. Now, in real life, if you wanted to know why for example this instance didn't get launched, if it didn't get launched, you click on the cogwheel on the right hand side, you scroll down and there is a state transition reason and straight transition message that we can have. We close it and basically this added at the very end two columns and so it tells us why the instance got terminated. So this one is because it was user initiated and then the straight transition reason as well, pretty much the same thing. But this tells you why these things happen. So let's see what this lecture really what's most important to you is to remember that if the instance goes from pending terminal is one of these four reasons that if it's a capacity error, it's not your fault, it's AWS. And if it's an instance limit error that means that you have too many instances and you need to open a ticket to increase your limit. OK, I hope that was helpful and I will see you in the next lecture.

6. Troubleshooting EC2 Launch Issues

Okay, another issue you need to be able to troubleshoots when you can't SSH into an easy to instance. Number one is you need to make sure that the private key file, the PM file does have the 400 permissions. Otherwise we'll get an unprotected private key file error and we'll see this in a second. If we connect to a Linux and we give the wrong username, we'll get a host keynote found error. So we'll do this in a second as well. And then finally, if you get a connection timeout and these happen, they're usually a security group issue because they’re not configured correctly or the instance CPU load is so high that we can't reply to us. So let's have a look with these three issues in the hands on. So I'm going to SSH into my first instance, that’s the only one I have left running and let’s go ahead and try things out. So first of all, I will mess around with the permissions of my file and so I will just assign wrong permissions to my PM file. Now if I try to do the SSH using my Pin file and copy the IP address, now we get an unprotected private key file error. That’s because the permissions are too opened and so because we have bad permissions, we can't login. So what I have to do again what I did before is Schmidt 400 and then we pass in team file and then we can finally login. Great, that one works. So that's issue number one. Issue number two is when your SSH into the machine but you don't provide the right username. So this EC2 user right here is provided because it says Amazon Linux Two. But if we think mistakenly that it’s Ubuntu, you know, who knows? And we do this and press Enter, then we get a permission denied and that's because we are logging into the wrong username. So that's issue number two and that's very common. Again, make sure you log into EC2 use rat the IP, not Ubuntu at the Piano this works. Finally, let's mess around a little bit with the security group and see how things work. So we are here and I'm going to go to my security group and as we can see right now, we allows from everywhere, but if we allow SSH from nowhere, so we’ll just remove SSH rule for example, or having none and we try to SSH into a machine. Now it just times out there is nothing that happens. And so when you see a timeout that either means that the security group rules are wrong just like right now, or that the CPU usage of our instance is too high, but we know that our instance is not doing anything right now. So the CPU usage is fine. So this time out issue really reflects one timeout and that time out is due to the security group rule being wrong. So, again, if you want to make this work, you need to add the SSH rule on Port 22 from cider that allows you to SSH into it. Simple, right? So now if I stop this and re-SSH, I'm in. Okay, excellent. So we've seen the three most common issues when we try to SSH. You have to remember them, but they're very common as a synopsis admin that you'll have. And I will see you in the next lecture.

Hide