New Domain 2 - Design for New Solutions

23. Kinesis Data Analytic Streams

Hey everyone, and welcome back. In today's video, we will be discussing the kinases of data analytics. From the name itself, we can figure out that the kinase data analytics will be able to do some form of analytics on the streaming data which it receives and it exactly does that. So let's go ahead and spend some time understanding what this solution does. Now, when we speak about the kindness firehose, the firehose basically aims to deliver data from point A to point B. Now, in such cases, the analytics is performed from the data which is received at the point B level. So let's say that you have the producers. Now, producers send the streaming data to the firehose. Now, Firehose in turn can send streaming data to S.

Three, you have Redshift, Elastic Search, and potentially other solutions. So from here, let's say that the firehose is sending the data to Elastic search. From Elastic search, you will be able to perform analytics on it, and you will be able to build dashboards from it. But this is the point B level; nothing happens in terms of analytics at the firehose level. Now, as opposed to that, when you talk about the kindness of data analytics, the analytics can be performed at the kindness level. So you're at the kindness level. There was no analytics. Certainly there was a small amount of transformation that could happen, but the analytics was at the point B level. However, as opposed to that with data analytics, the analysis can be done at the kinase level as well. So kinase data analytics has the ability to analyses the data streams in real time. So whatever graphs, whatever output that you might want, you can do it at the data analytics level as well.

You do not really need a third-party solution to achieve that. All right, now there are two supported languages for analytics. One is the sequel, and the second is the Java. So let me quickly show you what exactly this might look like. So, from the EWS management console, let’s go to the Kindnesses here. So if I click on get started, you have the data stream, you have the delivery stream, and third, you have the analytics stream. All right, so basically from this diagram, it says that are associated with each one of them. You will be able to understand what exactly this does. So basically, when you talk about the firehose, you have the producers, you have the firehose. Firehose is sending data to various endpoints here, and from the endpoints the analytics is generated. Now, as opposed to that, within the data analytics, you have the producers, you have the kindnesses analytics, and then you have the various dashboards here. all right?

So basically, when you go ahead and create an analytics environment, you will be able to see that there are two specific runtimes which it supports. The reason why it supports runtime is because we have already discussed that analytics has to be performed at the Kindnesses data analytics level and this is the reason why these are the run times. Now, for the exam perspective, you will not be asked about how you can perform analytics. In general, during an exam perspective, they will give you a use case and you will have to choose which of the kindnesses solutions can achieve that use case. Whether you have the data streams, you have the data fire hose, you have the data analytics and video streams. So you must understand which of the Kindnesses solutions can support which type of use case, and this is something that will be tested during the exam. So that's the high level overview of data analytics. Again, we'll not go in depth because understanding SQL and Java is not really important for the exams.

24. Kinesis Video Streams

Hey everyone, and welcome back. Now in today's video we will be discussing the Kindnesses video stream. From the name itself, it is pretty easy for us to identify that this kind of stream is specifically designed for the applications which will be producing and consuming video streams and streaming data. So let's go ahead and understand more about it. As a result, the Amazon Kinases video stream makes it simple to securely stream video to AWS from connected devices. So let's say that I have a camera not only on my laptop but on my premises as well. And I want to stream the data from My Camera into a central place where I can do a lot of things like machine learning, analytics, etc. Or so this is where the Kindnesses video stream really helps you. So it can be pretty easily understood with a diagram here. So let's say within the input you have the camera. Now definitely, whatever camera that you have, it needs to have the appropriate SDK. Now, the camera device will send the video stream to the Kindnesses video streams. All right. Now, from the kinases video stream, it can be integrated with the Amazon AI services. Now, AI services can do a lot of things, like let's say that a traffic cop is basically using the kinases video stream to check on drivers who are breaking the rules. So you have the camera here who is looking into all the cars. So now they have seen that there is one car which is overspending. So all of the data is going to the video stream.

So from the video stream you have the Disservices which basically looks into the license plate and from there you have the output where that specific car and its associated license plate can be displayed and can be alerted through it. So this integration is pretty great because it makes things much easier. And this is why, for the use cases where you have video data, the video stream is something that you should go forward with. So many of the individuals have an installed camera with a Raspberry Pi, they have integrated the SDK, and they are streaming data to the thinnest video stream. Now, there are plenty of good videos available. I'll be referencing one specific video which is pretty great after our current lecture, so that it will be easier for you to know how exactly the video stream would really work. Now basically, if you go into the kinases dashboard, the last one is the video stream. So let's click on video stream over here and you will see that the video stream is not really supported within the Ohio region as of now. So you will have to select one of the regions here. Let me just select Frankfurt for our testing purposes. All right, so we'll call it Kolas Hype and video stream. Now you can deselect the default settings, you can select media types, and you can also have data retention as well as encryption if you intend to do so. However, let's keep it as the default and let's create a stream. So now, whenever you create a stream, you will see that you will have a media player. So this is a media player's kind of functionality.

So, if you have a camera and whatever video stream it is sending to the kinases video stream, you can see it all live in the kinases video stream console. And this is really great. You can even back it up by 10 seconds, forward it by 10 seconds, and various other things. So this is a great function here. Now after this, you can even integrate it with the AI, which can do various analytics on top of the video stream. So that's the high-level overview. You can, in fact, even integrate this. Let's say you have a camera on your Linux or Windows laptop. You can make use of G streamer. You will have to do some kind of compilation there. Again, since this is not part of the official curriculum, we'll ignore it. But I'll link you to a nice video where the individual has shown how exactly this can be done and how exactly the stream from the camera is visible here. So that's the high-level overview. Now, keep in mind that in your exams, if you have any use cases involving video or camera data, analytics must be performed. So make sure you select the kindness video stream there. So that's the high-level overview. I hope this video has been informative for you. I look forward to seeing the next video.

25. RTO & RPO

Everyone and welcome back to the Knowledge Poodle video series. Now today we will be speaking about an important topic called RTO and RPO. And these are two factors that are very important, specifically when you are designing your architecture environment based on High Availability. So let's go ahead and understand what RTO in RPO means. Now, if you remember, high availability architecture should always be driven by your requirements. Now, whenever you design a highly available, fault-tolerant infrastructure that is also Multi-Availability Zone, there is always a cost associated with it. Now here, within the load balancer, there are two servers.

Now, I can always decide to have five servers over here that will basically improve my availability aspect as well. But I cannot always do that because if you try to increase your servers, the cost of your infrastructure will also increase, and the business is always after the team or, I would say, after the infrastructure team trying to decrease the infrastructure cost. So, whenever you design a highly available environment, you must ensure that it not only does not increase the cost to an exorbitant level, but also has a very good cost-to-availability ratio. Now, there are two important factors when you design a high-availability environment. One is the recovery time objective, which is also called an RTO, and the second is the recovery point objective, which is known as RPO. When we talk about recovery time objectives, we are referring to the amount of time it takes for your environment to recover in the event that your infrastructure fails. So let me give you one of the examples.

So let's assume that your infrastructure is located in the Mumbai region. Due to some natural disaster, something happened and the Mumbai region went down. Now what will you do? How long will it take for you to migrate your infrastructure to a newer region so that your business operations can continue? And to that question, the answer to that question is given by the recovery time objective. So let's assume that your RTO is 3 hours. Then you need to invest quite a good amount of money in designing a Dry region. So, if your main region or the Mumbai region goes down, your Dry region should always be ready so that you can migrate your traffic and domains to a new Dry region and continue your business operations from there.

So this is why RTO should always be defined. The next important topic is recovery point objective, which is directly concerned with the data maximum. It basically tells us the maximum tolerance period to which the data can be lost. I think it should be lost over here. Again, this is a very simple example. If RPO is 5 hours for a database, then you should be taking a backup of your database every 5 hours. So RPU basically tells us how much data can be lost as far as business is concerned, so if business says OK. If I lose the last 5 hours of data, it is fine with me. It is not a very big issue, but if I lose data from the past 12 hours, it is a major issue, so in such a case, if business says your RPO should be 5 hours, then you should be taking the backup of your database or important data every 5 hours. So when we talk about the relationship between RTO and RPO, this diagram can basically explain it. It's a very simple diagram to understand.

Your operation is going on and suddenly a disaster has stuck, so Arduous basically tells them how long it will take for you to recover from this disaster. Now again, if a Mumbai region is down and your entire infrastructure is in the Mumbai region, you cannot really rely on the crowd provider to make it up within your RTO time, so you need to be ready with a backup region, maybe a backup region in Singapore. So you need to be ready with the backup region and, depending upon the RTO, the overall preparedness for you to migrate your entire infrastructure changes. So this is the RTO again. It says how far back can you lose the data? Now again, RPO is very important in the RTO decision making process because when you are migrating a region, if your Mumbai region is gone and if you are taking your database backup, let's assume 10 hours back, so whenever you are migrating to the new region, let's assume Singapore, then you will have a database backup of 10 hours back. So that is something which is not really recommended because business will not really like it. Your RPO and RTO are very much interconnected and whenever you design a highly available environment, you should consider and you should design your environment in such a way that the RTO and RTO P O metrics are fixed.

26. Scalability with RDS Read Replicas - Part 01

Hey everyone, and welcome back. In today's video we will be discussing the RDS read replicas. So let's understand this with a simple example. Now, typically, if you go into a bank, there will always be different kinds of people for different kinds of work purposes. Now, this can be explained With the simple example over here, like in a bank, there would be a cash collector, there would be a check counter, there would be an inquiry counter, and many more. Now, in the case of this distributed setup, if there is only one person in the bank who does cash collection and who does the inquiry part, then the amount of queue within the bank will increase and that will slow down the operations. And this is the reason why it is important to have multiple people for multiple purposes. And the same applies to the database as well. Now, typically, if you make use of a single database for all kinds of activity, then it will slow down the overall operation.

So let's say that you have a single database where you have all the read, you have all the rights which are going on, and one day there's a huge amount of load, then one database will keep stalling and the amount of time it takes for the connection will also increase. And this is why a lot of organizations scale the database in such a way that a single database will not have all the operations coming towards it. So this can be understood with the example of having two databases which are set up. Let's say this is the master database and this is one more replicated database.

So now what you can do is, if you have an application which is present within your organization, it can send all the right queries to the master database. Now, since the master database is also replicating with one more database that you have on the right hand side, you can send all the read queries here. So this becomes a much better architecture because, let's say that you have a lot of read queries that are happening on your website, instead of sending the read queries to the master DB, you can send them to the replica DB that you have. And this will prevent the master database load from increasing. This type of architecture is now useful for organizations where the number of read queries is much higher than the number of traditionally written queries.

So one of the simplest examples that I can share where this type of architecture will be useful is our Wikipedia. Now, in Wikipedia, the amount of read which is happening is much, much higher than the amount of right. Now if I have to ask you, have you ever written a Wikipedia page or have you ever contributed to it? The ratio of the amount of read to the amount of written will be significant. There will be a significant difference over there. So, for Wikipedia, because the majority of the requests that may come in will be typed, this type of architecture will make much more sense. RDS now allows you to create read replicas much more easily. Again, if you have a MySQL server on your AC Two instance, you can do the same thing. But with RDS, you can do it in just a few clicks. So let's look into how we can do that. Where I have a DB called Kola Seven DB. It is present within the North Virginia region. Now again, this is a single database over here. And what we want to do is to have a read-only replica based database. So what I can do is I can click on this DB and if you go to Actions, there is an option to create a read replica. Let me create a read replica here and it asks for the destination region. Now, read replica can not only help you have a scalable database, but it will also help you in a disaster recovery fashion.

So let's say that this primary DB is present within the North Virginia region and, due to some reason, this DB goes down. Then if you have read replica DB in a different region altogether, let’s say Singapore, then even though the entire North Virginia region goes down, you still have the replicated data within a different region altogether. So not only does it help in scaling, but it also helps in disaster recovery. So the first option specifies the region in which you want your read replica to be located. So you can put it in a different region altogether. For our purposes of testing, let's put it in Singapore.

Now, also remember that this will only increase the shortage because they are in a different region altogether. So that is one important thing to remember. Now, if you go a bit down, the DB instance class is a very important parameter. Now, generally, what a lot of organizations do is, let's say, an eight GB RAM-based database. They only make the replica with two gigabytes of RAM. Now, this one is of a higher configuration, and this one is of a lower configuration, and this will impact the performance. Make sure that the read replica is always equal to the master database. So if the master database is of T two medium, then make sure that the read replica is at least T two medium, or it can be higher as well. So, if your organization has a large number of read queries, this can be T2 medium and even T2 X large.

So you can always have a read replica, which is much higher than the master DB. But make sure you don't lower the configuration of read replica when compared to the master DB. So I'll just make it T two micro. because my master DB has two micros. You can always increase the size of your read replica depending upon the requirements that you might have. Now, the next important configuration is the DB instance Identifier. So I'll just say kplabsdbhyphen replica and the rest of everything. We can just leave it as default, and I can go ahead and I can do a create read replica. So now the reading replica creation has been initiated. So typically, if you go to the Singapore region, you will see that your replica creation should be initiated. Let me go to Singapore.

And if you go within instances, you will see that there is a new DB replica called the Kpilab Seven DB replica. The status is created. Now, depending upon the amount of data that you have, this might actually take hours. But the master database from which we are replicating is just newly created, so we don't really have much data there. So, this should be fairly simple. Even with the base database where you don't have much data, it will take around five to ten minutes for the process to complete. So what we'll do is we'll make a computer video with this, and in the next video, we'll come back once the replica is created, and we'll look into a few more options related to the replica. So that's about it for this video. I hope this has been informative for you, and I look forward to seeing the next video.

27. Implementing & Analyzing RDS Read Replicas - Part 02

Hey everyone, and welcome back. So finally, our Read Replica status is now available. So it took around ten minutes for the status to be available. Do remember that it took ten minutes for a pretty empty database. However, if you already have a database that contains a lot of data and you are making a rate replica of that, then it might take a lot of time. Again, time is dependent on whether you are creating the database in the same region or in a completely different region altogether. In my case, we created one, with the master in North Virginia and the reader replica in Singapore.

So there is quite a difference in terms of distance between the Master and the Read replica. Anyways, once you have created it, we can go ahead and try it out to see if the replication is working or not. So in order to do that, let’s go to the master database. So this is our master DB. So let's do one thing. Let's quickly connect to the master database and create a sample database to verify whether the replication happens or not. So I'll copy the endpoint URL of my database and before we go ahead and connect to it, let’s verify whether the connectivity is present or not. So here it says that the connectivity has succeeded. If not, you must ensure that your IP address is allowed within the security group associated with the RDS. So now that we know that connectivity is there, let’s go ahead and connect to the database. So I'll say in my case, the username is KP admin, and let's do a password. great. So now that we are connected, let’s quickly do a show databases. And currently, there are a few databases that are available. Let's do one thing. Let's create a database.

I'll say create a database KP demo. All right, so the database is created and if you do a Show database, you would typically see that there is a KP demo database which is currently being created. So what I'll do is I'll log out from the MySQL and this time we will be going to the Singapore RDS and will connect to the MySQL Read Replica over there. So let's go back down and we will copy the endpoint. And as usual, let's quickly verify whether the connectivity is present. In my case, it has succeeded, and we can go ahead and try to connect it. So, since this is a read replica, the username and password will remain the same. Perfect. So now we are connected to the Read Replica. You can now do a Show databases and you will see that there is a KP demo DB which has been created. Now, one more important part to remember is that since this is a Read Replica, you will not be able to write within this database or within this instance. So if you try it out, let's say createdatabase, let's say KP write, and it gives an error saying that the MySQL server is running with the read-only option. So it cannot execute this specific statement. As a result, you can't really write on the replica. The right should always happen in our case. If you see, the write should always happen in the master DB, and the read replica will asynchronously replicate from the master DB. Now, a few more important parts to remember.

Specifically, when it comes to the cloud watch matrix of read replica, there is one matrix that you should always monitor and that is the replication lag. Now, replication lag is very important. If you have too much lag, then it might happen that your application will start to send stale data. So, always monitor it specifically if you have read the replica. This is one of the very important metrics that you should be looking for. So, with this said, let's discuss some of the important points related to RDS read replica for the exams. Now, generally, we already know that read replicas are generally preferred where there is a read-intensive workload. Now, in order to deploy a read replica, you must have automatic backups turned on. This is one important part to remember. And typically, let me show you this. If I go to the North Virginia region and let's click on the database, this is a Master db. And if I quickly do a modification over here, let’s go a bit down, and you will see that the backup retention period here is seven days. And this is the backup window. So, this is one important point to remember. Now, the third point, which is important for exams, you might get a question related to that, which is that you can have up to five read replica copies for the US database.

And if it is an Aurora, it can be 15 different replicas. Now, one more important point to keep in mind is that read replicas can be promoted to be their own full fledged database. So, let's say that let's consider a situation where you have a master DB here in the North Virginia region and you have a great replica in the Singapore region. For some reason, the Northern Virginia region went down and you are unable to access your master DB at all. Now, what you can do is you can promote this replica as the full fledged DB by itself. Now, once you promote this as a full fledged DB, the the replication will stop and this will act as a master DB. So, this is perfectly possible in RDS read replicas. And one last point to remember is that read replicas are available for MySQL, PostgreSQL, Maria DB, and Aurora. So, before we conclude this video, let me quickly show you some of the pointers that we have discussed in the console. Now, if I quickly go to the databases, we have already created one read replica for this capital as DB.

Now, you can easily create If you go to actions, you can also create one more read replica from this master DB. You can put it within the same region, or you can put it within a different region altogether. I'll show you one of the use cases typically during compliance, where data being lost is not tolerated at all, even at the compliance level. So generally, what we had was a database within the Singapore region, and during the audit, what the auditor recommended us to do, I would say it is a must do. The database copy that you have within the Singapore region should be backed up to a different region altogether. So what we've done is created a replica area from the Singapore region to the European region. So we always had two copies. One was within Singapore, which was the Master. And, if the Singapore region goes down or something happens, you always have a copy in the other region to access or investigate. So this is one part. One more important point that we were discussing is that Dreamtripnica can be promoted to be their own full-fledged database. So let's look into how you can do that in case you intend to do it. So if you go into instance, this is the read replica, and within the instance action, you see, you have an option to promote this read replica. So, if I click on promote this replica again, you'll see additional options, such as enabling automated backups. It's also asking you for the backup window.

Now, in case you just quickly do a continue, it is basically giving you an informational message saying that it is important that you stop the transactions on the master and wait for the read replica lag to be zero. We already discussed the importance of reading replica lag. Now, if you promote this as a master and there is a huge amount of read replica lag, since the connection will break, you will have certain transactions that will be uncommitted within the read replica. And hence, if you promote this as master, it will have certain delta differences when you compare it with the Master DB in the North Virginia region in our case. So once you are sure that you don't really have a read replica lag, you can go ahead and click on Promote Read replica. So now, this read replica is being promoted to be its own master DB. So let's quickly wait and we'll look into whether it has really promoted itself to a master DB by creating a new DB inside this instance. Because the last time we did it, it said you couldn't really do anything because this is a read replica. So let's quickly wait a moment. All right. So it took around five minutes, and now the status has been changed to backing up. So we can go ahead and try to run the same query that we had run earlier. Let me just execute it. And now, you see, it says that query. Okay, one row is affected. So, if you quickly do a show database, you'll see a new DB called KPI. So this is how you can promote the replica to be its own master. DB by itself, but definitely the connection.

Prepared by Top Experts, the top IT Trainers ensure that when it comes to your IT exam prep and you can count on ExamSnap AWS Certified Solutions Architect - Professional (SAP-C01) certification video training course that goes in line with the corresponding Amazon AWS Certified Solutions Architect - Professional exam dumps, study guide, and practice test questions & answers.