Designing Splunk Architecture

1. Splunk Visio Stencils usage

designing splunk. Architecture Learning about Splunk design before implementing enterprise level high availability multi-site clustering is essential because you will have a clear understanding of the size of the environment we will be implementing as part of our tutorial. Trust me, it will be one of those things you can say you have implemented because you will be part of each and every step right from the beginning of designing the architecture till the time we complete it and publish it on the Amazon AWS cloud. Before we begin designing the architecture completely, we must understand that architecture design without proper representation will not have an effective impact. To make Splunk architecture design impactful, we will have to do a couple of preparation tasks like having visual installed and making sure that we have visual tools installed and what each icon means to understand Splunk visual utensils. Or you can say that icons would have to go first. Let us download our vision stencils. Go to the first link. It is nothing but a wiki page. The second link is kind of a descriptive one, which is used for explaining each component in those stencils. It's a PDF document. Yes. In the first link, we can see Splunk visuals. It is in the Splunk wiki where it has been published and you can download it. As you can see, there are a lot of components listed. You might probably not pick it up now,but once we have done with our final course of publishing our enterprise architecture of Splunkon AWS, you'll be able to understand this. Most of these components let's click on these Visio stencils. We'll download the Visio stances while it's downloading and we'll go to, I believe, page number six of the icon collection. You'll be able to see all the icons within those videos, and tens what each icon represents. They have covered almost everything that Splunkarchitect would need to create an architect diagram. They cover almost every corner of Splunk implementationconsidering the batch, file input indexer, indexer clustering deployment,server, cluster master licence manager, all the components, including the heavy for order based on the OS level, as you can see here, every four orders for Windows, for Mac, and for Linux. These guys have done some extensive work in publishing this. But this has added a reality to creating the architecture. Let me open up my Visio. I'll create a blank drawing and we have downloaded stencils to add them. Click on more shapes, open stencils in Splunk documentation, if you need to unzip those files. Since I have already unzipped, this is the location, so I can click and click on Open. It will automatically add all your Splunk documentation icons. As you can see here, Let me expand it. See here. You'll be able to see all the different types of components using the architecture. Let me show you how easy it is to create a Splunk architecture using VGO. I am a user and my spiky architecture I'll create a simple architecture while keeping some minor details in mind. I'll do a couple of searches, not ten GB, but amedium size. Several firewalls are sending logs; let me add one more. Where are my indexes? I've had my searches. I'll add a couple of indexes This is one indexer. This is a group of indexes, so we can consider them multiple indexes. So what else do I need? Probably avoid it I'll add one forwarder, which isour universal forwarder client. I'll have one Linux forwarder and one Mac forwarder. These are nothing but the agents that are sitting on these servers. This is generic. This is for Linux. This is for Mac and we will add our Windowsforwarders. Also, it's a group of forwarders, usually the data sources you can represent is as one block. Let me put everything under a container and call this my forwards. That's it. So all this will send let me pull out a couple of arrow marks. Whichever you feel comfortable using, you can use them. Not the two sides. I'll use the one side. I'll create one. I've already created a couple of architectures. We'll go through them one by one. I'll create one for my searcher. This will be a two-way because I search and your index responds with the results similarly. When I say, there will be two responses from the user. The two ways are like a visual representation of the data, which you are squaring and he's getting a response in visualisation. This is the typical architecture. I know this looks ugly.

2. Estimation of License required

It's much more efficient and much more meaningful to create custom icons using Splunk icons.For this exercise, I've created three scenarios like Small Enterprise, Medium Enterprise, and Large Enterprise. And the last one is the crazy one,which involves high availability and clustering architecture. We'll go through them one by one. Before going to that, we have a few more things to sort out. Let's learn those things. That is the licence calculation, which is one of the crucial things in designing any architecture. The most important step in any Splunk implementation is determining how many licences are required. This is by far the most difficult step in designing the architecture because there is no straight answer saying I need 100 GB to 100 GB. There can never be a straight answer for how much data we are estimating from data sources because, as we all know, in some scenarios there will be logs because of an error or application crashing. We'll see how we can best estimate lot size in our environment. This step as a Splunk admin or architect needs you to interact with other teams and ask them what the log size or the data size was for yesterday. If they provide well and good. Next, ask them how many devices should be integrated with your Splunk. You will get a rough estimate. Keep that number. It's not over yet. You came from one team? Repeat the same step with other teams in the organization, like network for the Syslog inputs and flatfiles for either the system team or several teams for their data, and even the database team. After adding up all the numbers, let's say you come to the conclusion of 100 GB per day data. But based on my experience, it's better not to go for the exact figure of what we have calculated. It's good to take a 10% to 20% buffer so that any spiked logs should be manageable and should be well under our limit. Now, to conclude, after discussing and agreeing with all the teams, we can come to a rough estimate of probably 120 GB of data, including a buffer.

3. Evaluation : Search Head and Indexers

The next step after calculating your licence size is to identify how many indexers are necessary for your Splunk setup. Determine the number of indexes is difficult if you rely on Splunk documentation, especially official documentation. It says a single indexer can handle up to 300 GB per day. I think we all know the difference between the statements in official documentation of any product and the actual scenarios that as a consultant or end user you experience in the field. Based on my experience, it's good to have additional indexes for every 100 gigs of license. I've seen environments where one indexer is being choked to death by just not being able to handle 150 to 200 gigs of data because it was bombarded with applications, a lot of premium apps and stuff. Consider the search loads on the indexer. As per my own recommendation or my own experience, it's good to have one indexer for every 100 GB of data. For example, less than 100 GB should be enough,but greater than 100 and less than 200 or 250 GP go for two indexes, and anything greater than 200 and less than 300 GP go for three indexes and so on. Make sure that you have this in mind. This is just a rough estimate; there's no official recommendation, but trust me, when you go with this process, you will find it to your plan. Performance will be optimum. When I say optimum, it responds to you faster than having 300 GB of data in a single indexer. Moving on The next step is determining the search apps. Calculate the number of searches and, depending on a number of different factors, the length of the list varies. Let's say to list a few of them. The number of searches depends on the number of active users, the number of alerts or reports that are scheduled or real-time, the number of parallel searches, and the number of courses available to the searcher. Considering these kinds of factors, there won't be any clear answers at the beginning for you. You should consider this if you have more than eight users. Go for additional searches. So, let's say I have 15 users. I'll go for two searchers. I have 24 searchers. I'll go for three searches, so on. One searcher should be more than sufficient if there are fewer than eight users. Or one. Another advantage of search ads is that you can add them whenever you want. There is no need for any downtime or any impact on your existing Splunk environment, even the indexer. Also, you can add to your splunk environment at any point of time. There is no actual downtime or disruption in your environment. These are simple, small configurations which just scale your environment to the next level. The search apps can be added at any moment of time without any impact. However, while I recommend starting with a strong foundation of indexes, make sure you get the number of indexes right, because searches require a certain number of indexes. When you add, let's say the data will be shared between two indexes and the data storage will be inconsistent. In one index, you might see 100 GB of data. Another indexer, you can see ten GB of data. To avoid this kind of data storage imbalance, it's best to build a strong base foundation for your indexes. The next step in designing the architecture is to evaluate the need for heavy forward U.S. deployment servers. As we all know by now, these three components are optional components. We know what functionality these components are used for. So, as Splunk architect, it will be your responsibility to choose whether to have this component in the architecture or not.

4. Evaluation : Heavy Forwarder, License Manager and Deployment Server

Let's evaluate them one by one. Consider a V forwarder. There are three brilliant cases in which you can use a forwarder in your architecture. Number one is to filter out the locks. Let's say my firewall is just killing my licence because it is sending 200 GB of data per day. I'll filter out all the denied connections. I can filter out denied connections at the heavy forwarder level. And also, I can remove some event codes from the Windows event log to reduce the noise in my licence or the locks on my indexes. The first one is to filter out the logs. The second one is to mask your sensitive information from the logs. Let's say I need to anonymise some of the data which I'm sending to logs. I have some credit card information from my database which needs to be retrieved on Splunk to analyze,but the credit card numbers should be marked. You can do this using a heavy forwarder. Getting to the third point, having a heavyforwarder can add a major performance boost for your indexer. Assume your indexer is receiving 200 sysloginputs, iOS, or previous IOPSinput output operations per second. It's like gold for your indices. The more it has, the more efficient it is. When you are receiving 200 different log inputs from 200 different IPS on the indexer,your indexer is reading from 200 different sources. Let's say it is trying to read. For this example, we'll consider it committing 200 read operations for just receiving those logs,which is highly acceptable because I have only one left for processing, for storing, for fetchingthe results and giving them to my server. So what do I do? I place a V forwarder, intercept all this 200,reduce the noise initially, part the logs into pieces, then I'll feed them to my indexer. This will add a good performance boost and release some of your iOS from your indexor, which is done by your average time to go through all three cases where you need AV forwarders. One to filter out the noise in the logs, the second to mask sensitive information in the logs. Third, to boost your index or performance. These are the three cases where you can justify having or evaluating successfully having a heavy forward. The next one is evaluating the need for a deployment server and a licence master. Remember, the deployment server is a must in large scale deployment where you will be having hundreds of forwarders to manage along with your Splunk instance. The deployment server will be your friend, which can help in changing the configuration of a large number of clients. When I say clients, it can be your universal forwarder,it can be your indexer, it can be your searcher,it can be your average, it can manage all the clients and change the configuration in a matter of minutes. But if you have a small deployment of 10 to 20 clients to manage, there is absolutely no real reason for having a separate deployment server. If you're scaling up in the future, make sure you add a deployment server to manage your clients. And also, one point to remember is that our deployment server plays a vital role in managing the entire Splunk infrastructure and its clients. Now coming to the licence manager, this component, as we already know, keeps track of licence utilisation by communicating to all our indexes. In most of these cases, this is an optional step because the searcher, indexer, or your deploymentserver can perform this function. This licence master functionality is very minimal when compared to other components in most of the environment or organization. As you can see, it is usually clubbed along with your searcher or your indexer. The next and last option before moving onto storage calculation is clustering and High Availability.We will be discussing more about clustering in a separate module. As of now, we'll proceed with why we need clustering and Splunk. There are two main reasons for considering clustering and High Availability.Number one is availability of your data so that if any single instance of your indexer goes down,there should not be any impact or search results should be retrieved from one indexer. So that if I have two indexes in my environment, let's say I don't have clustering and high availability enabled, the indexer has like 50% of the data at any point of time. If one indexer goes down, I get only 50% of the results, which is not accurate. That is one scenario to go for a cluster. The second option is the integrity of data, so that the file system gets corrupted. You're not able to restore the data on one of the indexes. That shouldn't give you a chance to lose 50% of your data, but if you are clustering enabled, it should be available on other indexes. These are some of the major factors to know before designing an architectural project.

ExamSnap's Splunk SPLK-1003 Practice Test Questions and Exam Dumps, study guide, and video training course are complicated in premium bundle. The Exam Updated are monitored by Industry Leading IT Trainers with over 15 years of experience, Splunk SPLK-1003 Exam Dumps and Practice Test Questions cover all the Exam Objectives to make sure you pass your exam easily.