Amazon AWS DevOps Engineer Professional – Configuration Management and Infrastructure

5. CloudFormation Parameters

Now that we’ve seen resources, the second most important one is going to be parameters, and parameters are a way to provide inputs to your confirmation template. They are super important to know about if you want to reuse your templates across the company or, you know, other ELI accounts or regions, and some inputs cannot be determined ahead of time. For example, the key pair you’re going to link to easy to instance parameters is extremely powerful; it can be controlled, and it can prevent errors in your templates from occurring in the future thanks to types. So we’ve used parameters before in the first hands-on, in which we basically had to specify the security group description. So there was a string, and we were asked for the security group description, which was used in the security group itself. So you should use a parameter when, basically, you ask yourself, “Is this confirmation resource configuration likely to change in the future?” If so, you can make it a parameter, and by doing it as a parameter, you don’t have to re-upload a template to change its content. It’s a bit more stable and a bit more modular if you do programming, and I hope you do; you know the advantages of parameters. Now, parameters can have different settings and be controlled in many different ways. I don’t think the exam requires you to know all of these, but for me, and for your interest, I’m just going to name them. You can have a typing string number, a comma-delimited list of a type, and a list parameter. As far as description, you can have constraints, a constraint description, a minimum and a maximum length for strings, and a minimum and a maximum value for numbers. You can have default allowed values when you want to restrict the number of values. A user can pick an allowed pattern when you want to verify the inputs of a user using a regular expression and no echo if you want to pass in a secret, basically, so this is a lot of different optimizations and parameters you can do. For now, the only thing we’ve done is use a simple string parameter in our thing, and that’s all you should know, I think, for the exam. So how do you reference a parameter? In this case, you must employ the ref function. And so this is the first time we encounter references and functions. But basically, interesting functions in your templates allow you to supercharge it and link things up. And so the ref function is one of the most used, so you use it to reference parameters, and this way you can use your parameters anywhere in your templates. So look for any type of input, parameter, or configuration in your resources. So the shorthand for a reference function in YAML is “little exclamation point rest,” and this is why it’s just a way for us to have a much nicer syntax that’s easier to recognize. You can use the FN colonial and REF as well. It’s whatever you want, but most likely it’s going to be this exclamation point ref for shorthand. And the function can also be used to reference other elements within the template. So in the template we’ve been using in the parameters section, we defined a parameter named “security group description.” It turns out that if we look all the way down, the security group description was used in this security group description key, so we used the ref function to reference it. So this ref function basically says whatever the user will put as a parameter value for this parameter key, you want to use this and reference it into the group description. So it’s very simple to think about; it’s just a reference. But this reference function can also be used somewhere else. It was used right here, for example, for security groups. So you can see that this reference function also references the SSH security group that was created under resources. So the ref function can be used both to reference parameters like the one before and to reference resources. So here we reference the two security groups, and within the EIP we reference my instance. So as you can see, the name of the reference is the exact same name as the logical value under Resources. Finally, you have the concept of “pseudo parameters,” which are a list of available parameters that we can use at any time, enabled by default. And we only get a list of values that we might want to retrieve. So we can get an account ID, and you can just give us the value of the account ID notification, which has no value. If we don’t want to return a value, we can also ask for the region in which our advice confirmation template is being run at.In us east two, for example, we can also get a pseudo-parameter for the stack ID or stack name. Basically, you don’t need to know too much about these too much. Maybe the first account ID is very important to get the value of the account ID if you try to construct some complicated ARN value in your confirmation template. But overall, you should just know that there is this concept of pseudo parameters, and again, you can use them anytime; just use a ref and your old set. I hope that was helpful. I hope you know how they work now in your confirmation templates, and I will see you in the next lecture.

6. CloudFormation Resources

Let’s talk about resources. So resources are the core of your CloudFormation templates, and they’re mandatory. Your confirmation template cannot work without the resources block. And the resources, as the name indicates, represent the different apex components. Only a synonym will be created and configured, and the resources will be declared so that they can be referenced. so you can link the resources together. For example, you can link a security group and an EC2 instance. Any of us can figure out how to create, update, and delete resources for us, which is fantastic. And you should know there are over 224 types of resources, so I cannot teach you all of them, right? However, any resource must include the AWS, product name, and data type name. So usually, reading this identifier, you’re able to figure out what they were going to create. So, where do I go to find all of this documentation? Well, there is this great link that has all of them. So, while I can’t teach you how to find all of the resources, I can teach you how to find a link to the resources that does. So you simply read the document, and for example, we’ll read the document together just for the EC2 instance to get a feel for it. So the first web page I referenced to you is called the “AWS resource type references,” and this contains all the references that are supported by cloud formation. And the resource name naming convention is here. And if you scroll down, you can see there are a lot of resources. Honestly, you could count too many of them. So we can see that we can create a resource for pretty much everything. Let’s scroll down and see if there is something we know. Yes, auto-scaling; we understand how to set up an auto-scaling group. Using this confirmation resource, we can create auto-scaling group scaling policies and launch configuration. If we scroll down, we’re able to see how docode builds code that commits and could deploy a good pipeline. So all these things we’ve seen before, we’re able to do them. And if we scroll down, we can see there are two instances of the EC: the elastic IP and then the security group. So let’s take a look at EasyToInstance because that’s the one I believe we’re most familiar with. So the AWS EC2 two instances create an easy instance, which is self-explanatory. Now, basically, we can scroll down and look at the syntax of how to declare this simple instance. We have a JSON form, which is a little bit unreadable. But we scroll down and we get the YAML form, which to me is much clearer. So this is the YAML form for an easy-to-write instance. It must be of type AWS, be easy to create an instance of, and have some properties. So when you create a resource, it must have a type and have properties, and the properties are going to be key-value pairs. So, as you can see here, you can customise a lot of things for your EC2 instance. For example, if you wanted to customise the IAMinstance profile, we would click on this and get teleported to the documentation for that, and it would say, “Okay, you need a string, but it’s not required.” And if you update this, there’s no interruption. So that means that your EC2 instance won’t get terminated and recreated; it will just attach the instance profile to it. But if we change the image ID, for example, which is also a string but not required, then, in terms, if we update it, it will basically do a replacement. That means that it will terminate the old instance and replace it with a new one. So let’s go scroll back up. And so here we understand that there are so many things we can customize, and to know what we should specify, we should just look through the documentation. Now, if we look at what we created before, let’s just take EC 2 as an example as just EC Two.We have an availability zone, an image ID, and an instance type. And so, basically, I knew how to fill those out because if you click on AZ, then you see that you have to specify the name of the AZ where the instance is located. And so this is quite great, and you can just keep on going with this as much as you want. So now we basically understand how this was created. For fun, you could go into the resources here and see that we have an easy to use instance, but also that we have an EIP. So, if we look at EIP, we’ll go back one page and then search for EIP on this page. Here we go. We have EIP here, and we can see that the syntax is much shorter. We have an instance ID and a domain. And so the instance ID is what I have specified, but the domain is not specified. So here, when we go back to our EIP, we can see that, yeah, it makes sense. I did specify the type to be an AWS EC-2 EIP. And in terms of properties, I have only specified the instance ID. We’ll see what this reference means in a second. Okay, so the idea is that we are able to link the documentation to what we want to do. And so, obviously, for a security group, we need to provide the security group rules for ingress traffic. They’re an array because we can have multiple rules. So let’s go to the security group documentation. Just to prove that point, the security group is right here, and we go to YAML. And if you look at security group ingress, we can see there is a list of Amazon EC2 group rules. And so if we click on “easy to group rule” and we click on it, we go to YAML, and we get all the parameters that are available. So going back to the syntax, we get this little hyphen here that defines it to be an array, and here we get the first rule, and here we get the second rule. And so yeah, what we get out of this is that everything we configure through the UI can be written as code into your cloud formation templates. And this is how you know how to redact your cloud formation templates. So that’s it for resources. That’s honestly all you need to know. Just remember, there’s a type and there are properties, and all of this goes below the block called “resources.” So you need to make sure there is a small indent underneath resources every time you declare a resource. Now, the frequently asked questions for resources Can I generate a variable number of resources? No, you cannot. Everything has to be declared, so you cannot perform code generation, but you can have a dynamic type of code generation. And is every editing service supported? The answer is almost. There are only a few small ones that are not there yet. You can get around this by using AWS Lambda custom resources. Just a little bit of trivia here; you just need to know about it. We will not write to Customer Resources Lambda in this course. So that’s it for resources. I hope you enjoyed it; I hope that makes more sense now, and I will see you in the next lecture.

7. CloudFormation Mappings

Now let’s talk about mappings. Mappings are fixed variables within your confirmation template. They have to be hard-coded. They’re very handy if you need to hardcode some values based on the different environments you’re in. So dev versus prod or regions such as existing regions or AMI types, ETCA And as I said, all the values must be written out explicitly in your template. As an example, here is a map, and this is how you write it. So you have a mappings section, and then you have the name of the mapping. Then you have a key, and underneath you have keys called names and values. So it’s quite a low-level type of architecture. And so, to make it more concrete, we may have a region map to map regions to Amis. And so we’re saying, “Okay, within US East 1, UST West 1, or EU West 1, based on whether you choose a 32-bit or 64-bit type of architecture, here is the AMI ID you should be using.” So overall, this is just a hard coded “okay,” based on where the template is being run, and this is the AMI I want to use. So, when we use mapping and parameters, mappings are great when you know all of the values that can be taken ahead of time. So, for example, amid And so you can deduce them from variables such as region, AZ, list, account environment, etcetera, whatever you can think of. To me, they allow safer control over the templates. However, if you need the values to be truly user-specific, and the user must enter a value that you do not know in advance, you should use parameters. Now to access the mapping values, there is this function called “FN Find in Map,” which basically returns a value from a specific key. And the shorthand syntax is this one: We use Finding Map with a little exclamation point, and we have to give the map name and the top-level key and the second-level key. So we have three parameters right here. And so that’s something you should know for the exam, just the syntax of it, okay? And so if we look at this little cloud formation template, we can see that we have a regionmap that we have defined before. And so if we want to create an easy instance and reference the right AMI ID, then we use the Find in Map function for the image ID. And so the first one is the map name. So we’ll use region map because that’s what it’s called here. The second is that we want reference to the AOS region we’re in. So we’re going to use this pseudo parameter we just talked about and the ref function. So we reference the AIS region the confirmation template is running in. So, for example, say we’re running in USD Eastone, then we are in this block, and then we say 32 as the second level key. So we look at the 32-bit key and get the value from it. This AMI 6411-D-20 will be delivered to us. So this is the one that will be selected. So this is all you should know about mapping. Just remember the syntax of the find inmap function and the fact that mappings have to be written out explicitly in your templates.

8. CloudFormation Outputs

So let’s talk about outputs. And that’s actually a very popular exam question. So pay attention. The output section is optional, but we can declare optional outputs. And if we export these outputs, we’ll be able to import their values into other stacks. And by stack, I mean cloud formation templates. So you can start linking your confirmation templates. The outputs can also be viewed in the AOS console or via the AOS CLI. So it allows us to quickly retrieve the value of an output straight from the UI. In terms of example, for example, you can have a network cloud formation template, and then you would export outputs that would be, for example, the VPCID and the subnet IDs, and you would reuse those into other confirmation templates. And so it enables you to do cross-stack collaboration where you let the experts handle their own parts of the VPC and the subnet. And you, as an app developer, just reference these values out of the box. Something you should know, though, is that if you start using the cloud formation outputs and they start being referenced from another cloud formation stack, you cannot delete a stack that still has outputs being referenced somewhere else. So it’s just something to know. Now, if we look at an example of the outputs in this one, we are creating an SSH Security Group as part of the templates. And so we export that value as an output, okay? And basically, other templates will be able to get the value of that security group ID. So the syntax is pretty easy. We have the output section right here, and we have the name of the security group, and we have the description. So, for our company, SSH Security Group, here’s the value. So we provide a reference to the security group that was created within the resources. And then we have to specify this export block right here. Okay, this is an optional block. If you don’t specify it, then the value does not get exported, and it will not be able to be imported. So when we specify the export value, we can say, “Okay, this value, this SSH Security Group ID, is going to be exported as the name SSH Security Group.” So now, how do we import that value while using a cross-tack reference? And so we’ll create a second template that leverages the security group. And for this, we’ll use the FN import value function, which is an intriguing function, and we can’t delete the previous tag again until we’ve deleted all of this tag. So if we look at this little snippet of code right here, we can see that for the security groups, at the very bottom, there’s a shorthand syntax for the import value. And then we reference the exact same name as before, called SSH Security Group. So just a reminder: before, we exported the value as SSH Security Group, and here we import the value as SSH Security Group. So outputs and exports are a very popular question at the exam, especially if they start asking you how you link CloudFormation templates or reach the value from one to another. So you should know about it, you should know about the syntax, and you should know about the import value function. So I hope that was helpful, and I will see you in the next lecture.

9. CloudFormation Conditions

Finally, we’re going to talk about the conditions. So conditions are used to control the creation of resources or outputs based on some statements, logical statements, and the conditions can be anything you want. But the most common ones are that you might want to specify whether you’re in development, testing, or production and whether you want to create or not create that resource. Maybe it can be based on the region, maybe on the parameter value. As a result, each condition can refer to another condition parameter value or mapping, allowing you to compose them. So, to define them, to make it a little more concrete, here’s a condition I made under the block conditions. And so here we say, “Okay, do we want to create the production resources?” And for this, you need to have the environment type, which may be a parameter. This environment type requires a reference, so the value of this parameter is equal to string prod. And so here, basically, this whole thing is going to be true only if the environment is equal to production. And so using these productive resources, we’re able to basically define and condition other resources. So you can choose how you want to use it, and all the functions you can use in there will be and equals, if not or or. And so these are logical functions, and you can compose them as much as you want. Now, how do we use a condition? Well, you can apply it, as I said, to resources, outputs, et cetera. And so if we look at a resource, for example, at my mount point, the type of it is an AusEC, a two-volume attachment, and this only gets created if the condition that creates PROD resources from before is true. So that gives you an idea of how conditions are used, basically on the same level as the type right here underneath the name of the resource. So that’s it for conditions. I think they’re pretty advanced; I’m not sure they’re asked at the exam, but for you, it’s still good to know about them and that they exist, so you can have a little bit more logic in your confirmation templates. I hope that was helpful, and I will see you in the next lecture.

10. CloudFormation Intrinsic Functions

So the exam may ask you about interesting functions, and we’ve seen them. But here is a list of the ones you should absolutely know for the exam. The ref function, the get attribute find and map, the import value, the join sub, and the condition functions So let’s go ahead and have a quick memory refresher on what they are. The ref function is supposed to be the most important. It used to be referencing parameters, and if you referenced a parameter, it would return the value of that parameter. So we’ve seen this in practise when you provided the security group description and resources. So if you reference another resource in your confirmation template, it will return the physical ID of the underlying resource. So for example, if we reference an EC2 instance, we will get the EC2 instance ID. Now the shorthand for this is going to be exclamation point ref, and this is what it looks like. So here is a quick example. We create a subnet, and for the property VPC ID, we refer to the VPC that was created before, so what we get out of it is the VPC ID. As I said, when you reference an identifier, it returns to a physical ID. It’s super important to know. Now, how do we get more information from the resources? Because if we can only get the ID information out of the resources using a ref, then we’re pretty much stuck for some things. As a result, we can use get at, and attributes can be assigned to any resources you create. And so, to basically get a list of all the resources and all the attributes that get exposed from these resources, you will have to look in the documentation. So let’s take a quick look at it to get a feel for it. So if we look at our EC2 instance and go to return values, we can see that there’s a ref. So the ref will return the ID of that instance. Okay? And then, using the get attribute function, we’re able to get the availability zone, the DNS name, the private IP, and so on. So all these things are exposed. So to know what attributes are exposed in each resource, you have to go to the source and look at the documentation. For example, we saw that if we wanted to get the AZ of an easy to machine, we could do it. Here is our resource block. So we have our EC2 instance with an image ID and an instance type, and we want to get the AZ out of it. For example, when we create an EBS volume for this, we’re going to create a new volume under resources, and we’ll say the type is an EC two-volume condition if we had one from before. But the important thing here and there is that if you look at the properties, the size is 100 and the AZ is using the get attribute function of EC’s two-dot availability zone. So the EC-2 instance comes directly from this name on the left, and the dot tells us get attribute what we want to get out of it. So we’d like to obtain the availability zone. So this is quite a popular exam question as well. How do we get this attribute from this resource? And the solution is to employ the get at function. You have Find in Map, and we’ve seen this before. We use this finding Map function, and the shorthand syntax is that we have to specify the map name, the top-level key, and the second-level key. And so here is a quick summary. In this example, we were using the Find in Map function to find the image ID based on the region we were in and the type of architecture (32 or 64 bits). Import value is to import values that have been exported as outputs in other templates, and for this we use the import value function. So, remember, when we import a value, we simply give the import value name that was exported, and everything should work fine. Now for join, we haven’t seen it yet, but we can join values with the delimiter. So this is the shorthand syntax we should define for join. We give the delimiter name, and then we provide a comma-delimited list of values. To make it slightly more concrete to create the A column, B column, and C string, we can use the join function with the column and then specify ABC on the right hand side. So if you do programming, this is quite a common function in programming, so just know about it; sometimes you’ll be asked, “Hey, what is the output of this interesting function?” And if you use join, you know that you have to just put the columns in between the A and the B and the B and the C. Okay? Finally, we have the sub function.The sub function is shorthand for substitutes. It’s super handy; it allows you to substitute values within strings, so you can use it with references or pseudo-variables. To perform a substitution, the string must contain this dollar sign and begin with the variable name. So this is a bit obscure, but just remember substitution and it will totally make sense when you see it, if you see it in the exam. Okay? Sub is to substitute values. Finally, conditions are when we define, for example, a condition to only create the production resources, and we can use a bunch of functions in there, so the interesting functions we can use are and equals if not and or, and maybe others that will get added to confirmation over time. but that’s all. For the intricacies functions you should use, you will recognise that a function is called intrasync when there is this FN column or when you have that little exclamation point before a name that represents a function. This is all you need to know about CloudFormation functions. I hope that was helpful, and I will see you in the next lecture.

11. CloudFormation User Data

All right, let’s get advanced with cloud formation. Now let’s talk about user data. So we know we can have user data when an EasyTwo instance launches, and we can set it through the console. But there is also a way for us to directly include this in cloud formation directly.And so for this, we’ll pass the entire script through the function called FN base 64, which will convert our script to a base 64 string. That is what user data needs in CloudFormation. So the good thing to know is that when you run the script, all the logs are there, and that’s not related to confirmation but just user data in general. All user data logs will be stored in the Varlogcloudinit login. We’ll see this in a second. So let’s see how we can do this in cloud formation right away. Okay, so I’m waiting for confirmation. Make sure you’re in the North Virginia region as well. And there’s a new, redesigned abuse cloud formation console. It doesn’t change much, but we’ll just click on it just to make sure that it looks like something you may get to see in the future. All right, so we have this new UI. It’s exactly the same thing; don’t worry. So let’s make a stack so the template is ready, and then we’ll use and upload a template file. But before that, let’s go and see what the template is. So we’ll upload three user records in YAML. And so if you look at this template, it has one parameter, which is the SSH key we’re going to use. As a result, make sure you have a key pair on hand in US East 1. If it’s not created, then we’re going to create an instance, and it will be in US East 1A. It will use this AMI. This is why we are unable to use US. EastOne A is because AMI is region-based, and then we’ll have a key name reference key. So we’ll be able to SSH into our instance. And then we have an SSH security group that allows us to SSH on port 22 and also view on port 80 because we’ll instal a web server. And so the important thing here is user data. As you can see, we pass through the interesting function of base 64. We pass a script, and this whole script is preceded by a vertical pipe. So the vertical pipe here means that this whole thing is one giant string with new lines. If you delete this, it won’t work. So you need to have that vertical pipette say that all this is a script. And so in the script, it’s very simple. We’ve seen this before. We instal HTTPD, power on the system, and then type hello world into the index HTML. So that’s about it. Let’s go and have a look at how we can do it. So we upload a template file. So I’ll choose it and it’s user data, and then I’ll click on “next,” and the stack name will appear. I’ll just call it user data example and SSH key. If you do have a SSH key, it will show up here. Otherwise, you can go to EC 2. So if you go to service EC 2, then on the lefthand side of EC 2, you would go all the way to key pairs, and in key pairs, you could create a key, name it whatever you want, and then it will download it for you. So for me, I’ve already done that. So I’ll just use this confirmation key pair that I have. I’ll click on “next” and then I’ll just leave it at that. Click on “next.” The template is an s, and the parameter is SSH key, which equals a confirmation key pair. Everything looks good, so I’ll click on “create a stack.” So let’s go ahead. And now we’re just going to have to wait for our EC2 instance to come up and ensure that the EC2 user data gets applied. Okay, so it looks like my stack is now complete. So if I go to resources and refresh, I can see that there is my instance in the security group that was created. So I click on my instance, and they will take us straight to the EC2 management console, and in there we can look at the public IP. For example, here it is. When I visit the public IP URL, I get a Hello World from user data. The cool thing I want to show you is that we can SSH directly into this instance, and we can see that the user data gets executed by doing catVAR log and then cloud init log. And this is where all the user data output will go. If you press Enter, you can see that, for example, this was the command when we were running yum instal httpd. So everything got installed, and we got the output right here. Then, using systemctl start http, enable http, and echo LLS use from user data, it says cloud in it has completed at this time. So if there was any issue in our easy-to-use data script, we would look into that file again. We would examine the varg cloud init output log to determine where the error occurred. which is quite nice when you debug. But something to think about right here is that we created our stack, but our stack was not complete and working. Even if our user data did not work, we would still have gotten a good green tick. So we have to think about how we solve that problem, and we’ll see this in the next lecture. But for now, you’ve learned how to use user data with cloud formation. Congratulations. I will see you at the next lecture. And don’t forget to always delete your stacks right after you’re done.

12. CloudFormation cfn-init

All right, we’ve seen user data, but it’s pretty clunky as soon as you want to have something quite complex and quite structured. So for this, we’ll use CFN in it, and there is this block in the metadata of the resource that we’ll see in a second called Cloud Formation in it. And if you include it, you can use a script called the CFN in It script, which stands for “Cloud Formation in It,” and it helps make basically complex configurations easily readable, and the easy-to-instance will query our Cloud Formation service to get the init data and then apply it. And then, as soon as it applies it, all the logs will go to the VAR logcfninit logfile, and we’ll see it in a second. So why does it matter? Well, you’ll see in a second; I’ll show you. It looks quite a bit more readable than our script. So we have the confirmation service, and it will launch an EC2 instance. As part of the user datascript on this EC2 instance, I will run CFN, which will cause the instance query confirmation service to retrieve and apply the init data. So let’s see how this works now through an example. Okay, so as we can see in three user data YAML files, we had this giant script base 64, and it was a simple one, but it could get way more complicated. Think of all the things you want your EC2 instances to do when they start. So now if you go to CFN init YAML, we have the same parameters and SSH key, and we have an instance as well running the same AMI with the same key name and the same security groups. But now the user data is going to be a little bit different. It’s still a base 64, but we’re on top of it now that we’ve passed it through the sub function. We still have that vertical pipe, and the first thing we’re going to do is update the CFN bootstrap script to the latest version, which is something you should do because we use a Linux AMI, a Linux 2 AMI, and then we’re going to start the CFN init script. So we’re going to run optimist bin Cfninit.Then we pass in the stack ID. So this is a stack ID that represents a pseudo-variable in our stack. Then what we want to retrieve so my instance, wewant to retrieve the CFN in it metadata from myinstance which is right here on line 26.So we’ll retrieve this metadata, specify the region, which is a stack region, and then, in case of error, say “exit fail” to run CFN in it. So the metadata itself—oops, the metadata itself—is going to instal a simple Apache HTTP page, and it has the AWS confirmation init block, and in there there’s a config. And we can now read that configuration. I won’t walk you through how I wrote this. The important thing is to know how it understands the SIS Ops exam. So here we have packages, and we say, “Okay, from Yum packages, I want to instal HTTPD.” So here we’ve just said “instal HTTPD,” and we should do the exact same thing as when we did Yum instal HTTPD before. So we’re good here. Then there are files, and File is telling us that this file at VAR www HTML index HTML must have some content, which is all of these things. So here, as you can see, I had more stuff at the hone block, a p block, and it was easier because I could use that vertical pipe again, whereas in the user data script I had to use this echo thing, which was a bit clunky. So here we have more content, more freedom.We can also set the mode of the file to be equal to 64 bits. Then we can set some comments; we can say “hello,” and we’ll just run the comment “hello world.” And then, finally, we can say that services should start, and we say, “Okay, the HTTPD service should be enabled, and we should ensure that it is running.” And so the cool thing about this cloud formation is that I think it is way more readable than the user data script, which is a basic bash script. And so this is why we would use CloudFormation in it.The rest is the exact same thing. But let’s take a look at how this works in the console. So here’s the console, and we’ll go create Stack. The template is ready, we’ll upload it, and we’ll choose the file Cfninit YAML. I’ll click on Next, and I’ll call the Cfninit example and the SSH key; I’ll use the same key as before next and then create a stack. Okay, now our stack is created, and I’ll just wait a minute. Okay, so my stack has been created, and if I go to my EC2 management console and refresh and see all my instances, this one has been running. So I can go to the public IP and make surethat the CFN in it worked and it says hey, hello.Well, from the EC2 instance, this was created using CFN in it.It’s really cool that it’s working. But the cool thing I want to do is SSH into the instance and see what happened. Exactly. So let’s just go and SSH into this instance, and we’ll have to remove it after we receive the HTTP copy and paste mistake. Here we go. We’re inside of it. As I previously stated, the farlog cloud init is now available for inspection. And as you can see, we just ran this command CFN in there, but we didn’t really get any output out of it. We don’t see anything. So to see the output of this command, what you have to do is look at the filenamed VAR log or Cfninit log, and this file will contain the output of our CFN init script. So, as we can see here, it’s installed Httpd, it did the command “hello,” it enabled the service Httpd, and it started the success of the service successfully. So you can see a lot of things in this VAR log directory. You can see all the logs that come out of the CloudFormation things. As a result, it’s pretty cool. You can also look at this CFN init command log, which I felt could be interesting as well. But the idea is that you need to understand that if you needed to troubleshoot everything, they would be right here. And here we get the actual outputs of all the commands, which, if we wanted to debug them for whatever reason, we could. Look at this. So yummy, instal Httpd, it worked very fine andthen at the very bottom command hello world itsaid hey, hello world and it completed successfully sothat’s what I found really cool.Now we have more structure, and we have more files to look at in case things go wrong, and I think this is a better way to configure your instances. So the only problem we have is that this instance is still being created. So we have still create complete.But it doesn’t matter if our CFN init script works or not. We’d like to complete the creation process only after I’m confident that this has worked. So how do we do this? Well, I will see you in the next lecture.

13. CloudFormation cfn-signal and wait conditions

Okay, so as I said, we still don’t know how to confirm that the EC2 instance got properly configured and that it was successful after doing a CFN init. Maybe it failed; maybe we don’t want confirmation to complete because our configuration of our instance has failed. And so for this, we can use the CFN signal script. And so for this, we’ll run the CFN signal command right after the CFN init command, and we’ll tell the confirmation service to keep going or fail based on that outcome. For this, we’ll need to define a wait condition, and the wait condition will tell the cloud formation templates to wait until they receive a signal from the CFN signal. So we’ll block the template until it receives a signal and attach a creation policy that works on EC 2 and ASG as well. But the creation policy will dictate how many signals we want to receive and for how long we’re willing to wait. No worries, we’ll see the code in a second, so keep on hanging. But in terms of the diagram, you need to visualise how it works. We have the cloud formation service, and it will launch an easy two-instance setup just like before. This time, though, there will be a waiting condition. So the wait condition will just wait and put the confirmation service in hang mode, and it will wait until it receives a signal. So in the meantime, the EC2 instance will run the CFN init script, and we will query the cloud formation service to retrieve the initdata just like it did before. But after that, we’ll run the CFN signal script and signal the wait condition that we’re done, and then confirmation will be satisfied and we’ll be able to proceed. So let’s see how this works in practice. So in Cfninit YAML, as we can see, we ran the CFN init file, but that was nothing afterwards. So let’s go to the CFN signal and we’ll look at this. So the same parameters, the same MYAC, for example. But now if you look at the user data after the CFN init command, we’re going to run the CFN signal command, and it’s basically going through this little minus-e dollar question mark. So this is a special bash, but it basically says if this works, then send a good signal; if this doesn’t work, then send a bad signal, and we’re going to signal our stack and we’re going to signal our weight condition. So, in this region, I’ll go over the simple weight condition that was defined right here at the bottom in a second. And so this is saying that if this worked, signal the weight condition that it’s all good; if it didn’t work, still signal the weight condition that it didn’t work, and then cloud formation will draw its conclusion. The metadata itself hasn’t changed. It has the exact same cloud formation in it.So because it worked before, it should work now too. Okay, let’s scroll down. And here is our weight condition. So our weight condition for the type of cloud is a cloud formation weight condition. And the creation policy states that we are awaiting a resourcesignal, and the timeout will be, and I’ll just set it to two minutes to be fair. So PT two minutes, that means I’m waiting for a resource signal for two minutes, and when I receive it, I’ll be happy. You can also specify accounts if you want to. You could say count two, but that would make no sense because we’re not sending two signals. That is, if you create multiple easy-to-create instances for us, it could be counted as one because we’re just waiting on one instance. So I’ll just leave it like this. So this sample weight condition is waiting for one instance to signal something within two minutes. And that signal should undoubtedly be positive. And we know that the signal has been sent to it through the command right here. It says “resource: sample weight condition.” The rest is the same. The SSH security group has the same rules. And so the only things we’ve added to this stack are the weight condition and the CFN signal script in our user data. So let’s take a look at how this works. So we are in confirmation. I’ll create a stack, upload a template file, and this time I will upload Cfnsignal YAML. Click on Next, and I will just say CFN Signal Example, and I’ll just remove the typo because that’s pretty bad. Okay, now the SSH key is going to be the same as before. Click on Next, and on the bottom, click on Create Stack. So now the cool thing is that I will not pause the video because I want to show you exactly how things happen in real time. So we’re waiting for things to happen. The SSH Security Group is making great strides, and so is the sample rate condition. Now we’re just waiting for our EC2 instance to start up. So my instance is now in “Create in Progress.” So if I go to my EC2 management console, I should see my EC2 instance that is starting right now. But as you can see, my wait condition is still in “Create In Progress.” So it’s still waiting. So I’ll refresh again and wait for it to happen. Okay, so my EC-2 instance is now running, and if I go back to confirmation, this is usually when things will end. It would say my instance is complete. So it’s going to show this in a second. Create Complete. Here we go. My instance. Create complete. And so the stack is still not completed because the sample wait condition is waiting. Returning to stacks, we can see that Create is still active. So it’s because the wait condition is waiting for a signal. And so this will not happen until the EC Two instance has configured itself and uses the CFN signal. So I’m just going to wait a little bit longer for the CFN signal to arrive. And here we go. The sample weight condition has received a success signal with this unique ID. And so that was about 20 seconds after my ECTwo instance had been created, and because it received a proper success signal, we went into the create complete stage. And so that’s the cool thing. The condition had to wait 20 seconds for it to complete. Now, if I go to my EC2 management console and open up my public IP, I will know for sure that it has worked because the CFN signal sent a positive signal for confirmation. That is a really cool thing. Now with weight condition, we are able to basically say, “Wait a little bit until you can say, “Okay, your stack has worked,” because this easy instance will be able to send a signal right away in the next lecture.” We’ll see what happens when things fail, because that’s a very important test case for the examination. But that’s it for now. We’ve just seen that a wait condition allows us to wait for something to complete before we receive a signal. As usual, I will go ahead and delete my stack, and I will see you in the next lecture.

14. CloudFormation cfn-signal failures troubleshooting

You must be able to troubleshoot why the wait condition did not receive the required number of signals from an Amazon E2 instance, which is one of the most important questions on the exam. So number one, it’s maybe because the AMI you’re using has the confirmation helper scripts installed but not yet activated, so you need to make sure that it has them.Obviously, if it doesn’t have the help of scripts, you go online to the documentation and figure out how to download them to your instance. Then you need to verify that the CFN init and CFN signal commands were successfully run on the instance. And, as we can see, there are logs that help us debug the instance launch, such as the VAR log, cloud init log, or varg Cfninit log. Very important. Then we can retrieve the logs by logging directly into our instance using SSH. But as we’ll see in this lecture, we won’t be able to do so until we prevent a delete instance on failure. So we’ll see this in the next lecture, but in this lecture we’ll see how things would not work, which will be quite cool. And then finally, a very tricky one is that your instance must have a connection to the Internet. And so if it’s in a VPC, basically our instance should have internet connectivity through a NAP device if it’s in a private subnet or through an internet gateway if it’s in a public subnet. And we’ll see this in detail when we do the VPC section. But the idea is that if our instance cannot talk to the Internet, it cannot talk to the confirmation service. And if it cannot talk to the confirmation service, well, there is no way for the confirmation service to receive a signal, and things won’t work. So, for example, if you want to know if you have access to the confirmation service from the instance, you can use the AWS Amazon.com curl command. Very simple. So let’s go take a look at how we can trigger a failure for the weight condition. Okay, so in CFN signal failure, YAML, it now appears to be the same thing, but the failure is that my command hello will say boom, and then we’ll exit with the error code 1. And so when you do exit one, that means that is a bad exit code; the good exit code is zero; one is a bad one. So this will basically trigger a CFN init failure. And so we will signal a failure in cloud formation. And so let’s go see what happens in that case. So we go to our stacks, we create the stack, and we’re going to upload our failure stack. So it’s number six. Here we go. And click on “next.” Now Azure says that in the event of a CFN signal failure, the SSH key will still be the same. And next, and all the way to the bottom, create a stack. All right, so now we are going to have to wait just a little bit, and I’ll pause the video for the EC2 instance to get created and to report a failure. As we can see, my instance is now in “Create in Progress.” So if I go to my EC2 management console, I should be able to see, yes, my instance running. So now we should be calling CFN in and waiting for it to happen. And so my simple “wait” condition is still in progress. So it’s waiting for a signal, basically. So now I’ll just refresh and wait a little bit. My instance is now created, so CFN init should be running within a minute. And here we go. We get a “failed to receive one resource” signal within the specified duration. So I think that was even a timeout for this one. not even the CFN signal pinging it. So we fail to receive a signal within the specified duration. So Create failed for the sample weight condition, and as a result, my entire CFN signal instance and Stack are going to roll back. Rollback means that my instance, as well as my sample weight condition, will be deleted. So now if I go to my EastManagement console, my instance is shutting down. And so the cool thing is that, yes indeed, if the CFN signal doesn’t work, or if the CFN in it doesn’t work, or whatever, then obviously the wait condition fails, we’re happy, and we should roll back. But now the little problem is that I can’t debug why it failed because, as you can see, my instance is shutting down. So that’s a bit of a problem for me, isn’t it? because I can’t SSH into it. So we’ll see in the next lecture how to do this. But the cool thing is that now we have a sample weight condition that fails. And so we were able to debug why no resource signal was being received. It’s because there was an error in our scripts. So I’ll see you in the next class.