Designing Splunk Architecture

9. Large Enterprise Architecture review : Part 2

For simplicity's sake, this module. In this slide we'll be considering single side index or clustering. In single-side indexing or clustering, each index exchanges the data that it has received from the forwarder, so that at any point, if one indexer fails, the data can be retrieved from the other two instances. To understand how this actually works, we need to know more about replication and search factors, which we'll be covering at a later stage because of the complexity involved in choosing them and the depth of the concept of application and search factors.

Moving on Now we understand that single set clustering in Splunkexchanges data between indexes in order to overcome any data loss or disruption in our search results during the event of any indexer going down. Now we've got that clear. Let's move on further down and compare this with our Medium Architecture.If you see our medium architecture, this is our medium architecture. You can see that there are some components like the Deployment Server and Heavy Forwarder and License Manager. There are three blocks which are new in Large Enterprise.As you can see, these three components are present in the large enterprise of Splunk architecture. We all know by now that to expand the deployment of Splunk, we must have a deployment server for managing Splunk, Universal Forwards, and other Splunk components.

By having a single deployment server, we will help to reduce the complexity of managing Splunk infrastructure. Consider the example of having 200 clients and logging them individually just to change their anIP or the host name of that instance. Instead of that, you can push this configuration from the deployment server, probably in five minutes rather than logging into 200 different machines one at a time. The deployment service acts like a boss in an architecture where it talks to all the Splunk components across the architecture and tells each how to operate and what configurations are deployed on each component. We can now use Deployment Server to monitor the health of the components throughout the infrastructure. Now we understand the deployment server. Let's move on to our License Manager where we see from the architecture it keeps tabs on our licenceutilisation and can also alert based on licence violation upon reaching our thresholds as perour alert configurations for this licence usage. This functionality of the License Manager is almost identical to that of a non-cluster or clustered environment where its only function is to collect the licence usage fromall the components and keep track of the violations in the daily usage statistics of our licencesandreport them accordingly.

And we also see in this architecture that there are heavy forwarders in between UniversalForwarders and our data sources and indexes. These heavy forwarders are receiving the data from forwarders and passing it along, filtering some of the events before sending them to the indexer, where the indexes will be receiving the data from. These heavy forwards, which are passed and filtered by the heavy forwarders, are received on the indexes and stored on the indexes themselves. This is a good option and best practise to have an AV forwarder in your architecture of large deployments. It will add significant value to the Splunk architecture. By now we are familiar with the three architectures for small, medium and large enterprises. Now let us see one of the best architectures of Splunk, considering the scaled up version of large deployment.

10. Understanding clustering and High Availability in Splunk

This architecture can be considered as a scaled up version of the larger deployment which we saw in the previous tutorial. This will be one of the crazy things involving things like high availability and clustering of Splunk into your design. Since we've already gone through these scenarios of having high availability and clustering options, by now you should be aware of the benefits of having high availability and clustering options in your organization. Let's see the architecture now. Looking at Chile, it looks like a total chaos in the architecture and a lot of components, but as a Splunkarchitect you'll be able to see the beauty of Splunkflexibility and scaling up and its design. If you look carefully, there are two sites, which are Site One and Site Two. These are two sites.

In real scenarios, it will be like the main data center. This could be your doctor or Visa Recovery center. For our understanding, let us call them Site One and Site Two. The Site One components look identical to the last enterprise architecture, which we saw in our previous example. This is our site. One architecture If we just see in our previous discussion, we went through the large enterprise architecture, which is identical to our site one. It is clear that for Hillary Clinton and Cluster, we are considering only large-scale enterprise. So site one is our main data center, where all of the logs are collected using universal forward essays logs and then passed to the indexer for storage and retrieval by our AV forwarders, and the searchers do their fancy stuff like fetching data from the indexes and visualizing reporting or alerting. The same is true for Dr. or our sites, which are identical to our main site. But from this diagram we can see that some of the components, like the deployment server and the license manager, are communicating to both sides. Having a deployment server talk to all the components has a huge advantage of managing the configuration in one place.

It talks to all the components like searches, indexes, avoidances and the data sources. Similarly, we know from our previous modules that License Manager talks to all the indexes that are present in Side One, Site Two, or in any other site of your architecture to keep track of the license utilization. Since it has very limited functionality, we can make it a cluster master. Also, we can use the licence server itself to function alongside as a Cluster Master, which takes care of making sure that the data has been copied or replicated to the other side and vice versa. The function of Cluster Master can be clubbed with that of a deployment server or License Manager. Although it is not recommended by Spelunk, it doesn't have much of an impact on the performance. Since the License Manager, which has very limited functionality, can also be made a Cluster Master, it is also the Cluster Manager's responsibility to ensure that the replication and search factors are met between the cluster or the cluster members and that the cluster is stable. The health of the cluster can also be monitored from the cluster master. To conclude, let us go through some scenarios where multisite clustering will add value. Let's say one of the indexes in my main index goes down. So what happens? There is still data between the two indexes, which should be more than enough. If you have configured the replication factor, two will come to this replication factor and such factors and how they influence the cluster and the storage and the high availability part. Let's say we have two copies of the data here.

So if one indexer goes down, there is a very good chance that these two indexes can give you the results without any impact. Let's say one of the searches goes down as a second scenario. If it is a highly critical one and it is clustered into ourDry, we can access our Dry searchers and continue with our dashboard reports or alerting. Whatever it was, it should operate without any issues. Similarly, if it is a dedicated searcher, like it handles premium map which is configured only on one searcher and it has not been clustered, the impact will be the alerts or the scheduled searches which are configured on this searcher will not be running anymore. If it has been clustered into our global site, it will be subject to scheduled searches and alerts generated by our searcher on the site. Two. In the third scenario, let us consider there are two indexes going on. In that case, our search will be impacted. We will not be getting 100% of the results from the main site indexes, but if we make the same searches point to these indexes, they will be able to retrieve 100% of the data even though these two indexes are down.

So at any given point of time, either these three indexes or these three indexes should be able to serve you with 100% of the results. And in the fourth scenario, the deployment server goes down. Consider the deployment server goes down, which doesn't have a slave in this architecture, like it doesn't have a failure. But for the deployment server, there is a reason why it stands out from the regular architecture. If you see it, it stands somewhere in the middle just communicating to all the servers. But if you see, if the deployment server goes down, there is no functional impact on our Splunkarchitecture because it just makes sure that all the instances are up and you'll be able to modify configuration, restart them, and make sure the new configurations are deployed. These kinds of scenarios whereas even if it fails, the searcher indexes and heavy folders will have a local copy of their configuration and will continue to function normally. In the case of a deployment server going down, let's say you're not able to bring up the server. Make sure to restore the backup into the new VM and you'll be able to assign the same IP and the deployment server should be up within a matter of no time. By understanding all this architecture and the benefits, you should be able to design one of the best fit architectures for your organization.

11. Hardware Requirements for Splunk Architecture

As part of our journey of designing the best architecture for our organization, The next step is to understand the hardware specifications required for our Splunk components. The link specified here in the document should be able to take you directly there. Let me show you the contents of this so that you will have a better understanding. These are the hardware recommendations that are recommended by Splunk. The link should be able to take you directly into the requirements page, which shows the recommended hardware specification. These are for the Unix operating system. Now let us go through them one by one. Let us start by looking at the search perspective. Depending on, let's say, small, medium, or large enterprise. The number of courses for Splunk varies from two to 64 courses at 2GB depending upon the size of the architecture.

It's like a twelve-course for small enterprises and a 64-core for large enterprises. Because each core or it is core intensive, the more courses the better for the search. The search ads are displayed whenever you run a search. They mainly rely on the available courses for that search. It's better to have a higher number, of course, for our searcher and looking at indexer hardware, it is highlycritical to get a minimum of more than 100 IOPS. The IOPS should be more for the indexes since the more IOPS, the better the performance of your indexer. Always remember never to compromise on IOPS, that is your input operations and input output operations per second, since it is one of the critical values for the performance of your entire Splunk environment. Moving on to the next value is storage. From our previous discussions, we know how to get an estimate of the storage for our indexes. Now we need to understand what rate level is required or recommended by Splunk to run at optimum performance. It is highly recommended to have rated for better performance,but if you are able to get our IOPS condition,we should be fine with rate five or rate six.

The next step is the RAM specification, which depends again on the size of the deployment. Depending on whether the system is small, medium, or large, the ramp can range from twelve to 64 GB, similar to the course we discussed earlier. For the scale of the deployment, it's always better to go for the maximum available ramp. Since you will notice, Splunk will be acting like a monster. It will be eating up all the resources that it can get its hands on. And this can be tuned to run at optimum performance by a Splunk admin or Splunk architect. And also, there are a couple of prerequisites for Splunk which should be taken care of as part of infrastructure provisioning or before installation. Those are U limits as per Splunk recommendations. At the OS level, there are a couple of limits that need to be specified so that our Splunk operates at optimumperformance and also SELinux, also known as Secure Linux. On the Linux platform, it should be disabled or should be made to allow Splunk to run outside of Linux and PHP, which stands for Transparent Huge Pages, which is known to cause issues while running Splunk. So it is recommended by Splunk that you disable these processes before installation.

12. Capacity Planning for your Architecture

The final step in concluding the link specified in the document should be able to take you to the official documentation where you can download this manual, which will be handy while finalising the architecture. Let us go through this link. So this is one of the links that are very useful while you are at the final stage of your Splunk architecture. This manual is known as the Capacity Planning manual. You can click to download this manual as a PDF. Make sure you're clicking on the top because if you download this, you will probably end up just getting the first page of the documentation on this topic. So make sure you click on Download ManualAspedia so that you get the complete manual. So this is our Capacity Planning Manual, which will be very handy while finalising our Splunk architecture. And we have seen in our previous discussion what the licence set, number of indexes required, number of searches, number of AV formers, whether to have a deployment server, licence manager, and also the hardware requirements like RAM, CPU, IO required for each component of our Splunk, the storage requirements for indexes, and IOPS. We will summarise all this and finalise the best fit architecture for the organization. Remember always that IOPS should be greater than 200. The RAM can vary from twelve to 64 GB based on the size of the architecture and, of course, the better for the searchers.

Prepared by Top Experts, the top IT Trainers ensure that when it comes to your IT exam prep and you can count on ExamSnap Splunk Core Certified Power User certification video training course that goes in line with the corresponding Splunk SPLK-1002 exam dumps, study guide, and practice test questions & answers.