SPLK-2002 Splunk Enterprise Certified Architect – Post Installation Activities Part 4

9. Splunk Lookups

Hey everyone and welcome back. In today’s video we will be discussing about Splunk Lookups. Now the Lookup Knowledge object in Splunk is something which is very widely used and typically if you are working in an environment where Splunk has been implemented, you will find that there are a lot of use cases where the Lookups will be required. So let’s go ahead and understand the Lookups functionality in split Splunk. Now going with the definitive terms, Lookup basically enhances the power of Splunk by enabling correlation of search results with third party data like databases, directories, CSV files and others.

Now, in very simple terms, it basically allows us to correlate our Splunk data with the external information which can be in a database, CSV files and others. So let’s understand this with an example. So let’s suppose that there is an event within the Splunk which basically contains the Customer ID field. So Customer ID can be like 010203, but just by looking at the Customer ID you’ll not be able to get much more meaningful information. So maybe you might want to know what is the Customer name associated with Customer ID and various other information.

So Lookup will be able to tell you what is the Customer name when you have the Customer ID field. Obviously now Customer Name needs to be passed through CSV file or through other mechanism. But Lookup is the functionality which basically allows us to do that. So with this let’s go ahead and do things practically so it makes much more better sense. So within a tutorial data we have two files here. One is Lookups hyphen sample zero one and second is Lookups.

Now if you look into Lookup sample 15, this is a sample data here. So you have one purchased microphone on 24th August 2018. So this field is basically the Customer ID and then you have the product name and then you have the purchase date. So this is the sample file over here. Now, if you see just by looking into the Customer ID, we do not really know who that person is, all we know is the Customer ID. Now, within the tutorial data I also have a file called Lookups CSV. So if I quickly open this in WordPad, you will see that I have two fields. One is Customer ID and customer name. And within Customer ID you have zero one, zero two, zero three, zero four, zero five and on the Customer name you have ZEEL, Harsh, Supra, Thik, Yash and Amkit.

So we’ll be importing this data to Splunk and we’ll be doing a Lookup so that we know that zero one is basically zero, zero two is Harsh, zero three Supra Fake, zero four is Yash and zero five is Jumpkin. So let’s look into how it would work. So within Splunk, let’s go to settings and we’ll quickly add a new data. So this time I’ll add the Lookups hyphen sample one which contains our data. I’ll select next. So currently you see there is no breakage of line. So within the event breaks we’ll break at every line interval so that we have an individual event. So there are in total five events over here I’ll do a Next. So let’s save this source type as Lookups test and I’ll save it.

So we’ll go ahead and we’ll click a review and submit. So this is our sample data. However, within the sample data you see that the fields are not really parsed properly. So we’ll make use of the interactive field extractor functionality and we’ll quickly parse these fields with the help of space as a delimiter. So we know that this five is basically the Customer ID. Then field three is basically the product name. Then August is month, 24th is date and last one is the year. So this makes sense. We’ll go ahead and we’ll put this extraction name as Lookup. You can give it any name that you intend to and you can go ahead and you can do a finish.

So now if you go to Splunk surgeon reporting app within the data summary, the source type is Lookups hyphen test and now you see if you just expand this event, you have a proper field extraction. However, let’s say Analyst is looking into this log file. Now from this, only thing that he’ll be able to identify is that someone from Customer ID Five has purchased the scootie on 24 August 2018. However many times you might need to have more information, say Analyst also wants to know what is the name of the customer who is associated with the Customer ID. So one way is that your analyst needs to have maybe read only access to database. He’ll create a query to find the name associated with this Customer ID and he can get the name.

However, that is a trivial approach and you need to have multiple things, multiple access that is needed. Ideally, what you need to do is from the single interface, if you can get all the relevant information that is the best bet and that is the idle thing that is needed and this is the reason why lookups are very important. So in order to have the lookups on what we’ll do we’ll go to settings and on the left hand side we’ll select Lookups. Now with the Lookups, let’s create a new lookup. The destination app would be Search. We’ll select the Lookup file basically which is the Lookup CSV. So this is the Lookups file and at the destination file name, let’s say that it is Kplabs lookup CSV.

So this is the destination file name which will be stored. I’ll go ahead and let me just remember this, otherwise I’ll quickly forget it. Perfect. So your lookup has been successfully uploaded. Now if you search here you will see that the path where your lookup is stored is Opsplung etc. Users admin search lookup kplabs lookups CAC now, since I am logged in as Admin. It has stored inside the admin user directory. So now once you have your lookup file present, what we can do is we’ll go back to our search and reporting app and we’ll select the source type which is lookups underscore test. Now, what we need to do is we have to do the mapping of customer ID of this event to the event which is present in the CSV file. So here in the CSV file every customer ID is associated with a given name. Now within Splunk event, by looking at the Customer ID we need to know what is the associated name.

So in order to do that, what you need to do is you have to pipe it to lookup command and you have to basically put the name of the lookup file. So you have lookup kplabs lookup CSV followed by you need to give the name which is Customer ID which is present in the lookup. I’ll say Customer ID as now you have to give the field associated with the Customer ID which is present in your event which is Customer ID. So your ID is capital. So I’ll say customer ID. And that is it. Now, if you open up any event here you will see that there is a customer name field which has come.

So now this customer name is unkill. Let’s try once more. Let’s try zero one and within zero one you will see the customer name is ZEEL and all of this data is actually coming up from the lookup which is KP labaps lookup CSV file. And if you know, this becomes much more easier so if analyst is going through the file it becomes much more easier for him to understand what exactly is going on and it becomes much more simpler here. So with this, what I wanted is let’s take one more example. So many times what happens is that in the add ons and app that in add ons basically if you download it, those add ons they come with a lookup file by itself.

So let’s try it out. Let’s download the Apache add on and we’ll go ahead and we’ll do a restart with this we’ll quickly log in. Now, once we have logged in, I quickly wanted to show you is that now that we have downloaded the Apache app now if you go to etc apps and let’s do LS, iPhone L, where is the Apache? This is the Apache add on that we have. So we’ll go to Splunk Pache and if you do a LS, iPhone L there is a directory college lookups. So now if you go to Lookups you will see that you have Apache underscore http status CSV. So this is the lookup which the add on is coming with. Now, if you open up this file you will see that you have status, status underscore description and status underscore type. So you have various status 100 200. So basically Http has various status code and it is actually difficult to remember each and every one of these status codes. So for analysts it, it might be easier that if they see a status call as 20 five within the Apache or NGINX log for them it will be much more simpler if they also have the description on what that status is all about.

So let’s try it out on how we can achieve that. So what we’ll do is we’ll go back to our Splunk search and reporting app, we’ll go to the data summary within the source type we have access combined test, let’s put it as alltime and along with that within the settings we’ll go to look up. Now if you open this up, you will see that every log has a specific status. It might be 200. So if you quickly do stats count by status, you see there are a lot of status and for analysts he might not really know each and every status. So what he might do is he might do a Google. So if you have a 40 six he might have to do a Google on what exactly 40 six mean. Now, if you have the description associated with the status within the Splunk itself, so it becomes much more easier for anyone to read. So hence what we’ll do is we’ll go to settings, we’ll look into the lookup table files which are already present.

So there are a lot of Lookups which comes with Splunk. And this is the lookup file that we are interested in which is Apache underscore HttpStatus CSV. So we’ll copy this and let’s create a new query which is Lookup you have Apache underscore HttpStatus CSV. Now within the event the field name is Status and within the logs within the CSV the field name is Status. So first the first thing that comes here is the field name in CSV file and then you have to put the field name in your event logs. So now let’s run the query. So the query has been executed. Let’s open this up.

And now you see with status you have a status description of okay and the status type of successful. And now if we quickly verify you have status description field over here it says okay and the status type here is successful. So I hope you understood the benefits which Splunk lookups bring in and it really becomes much more simpler. So go ahead and try this out because this is a fun practical as well as this is going to be very useful within your Splunk career.

10. Splunk Alerts

Hey everyone and welcome back. In today’s video we will be discussing about Splunk alerts. Now, alerts are basically used to monitor and even respond to a specific event. Now, one important part to remember is that alert not necessarily mean that it would send you an email, however it does more than that. Specifically in today’s world alerts are associated with various kind of a script. So one example that I can share is that we used to have an alert related to Apache. So anytime Apache went down that alert was connected to a script and that script would automatically restart an Apache server. So it might happen many times that due to memory issues or due to other issues the process went down and quick restart generally helps a lot. So this is also referred as the event driven action where on a generation of a specific alert that alert is connected to a specific script and that specific script is executed when an alert is raised. Now, along with that throttling of alerts is also an important factor during outage. So one example is that let’s say Apache went down and your monitoring script is checking every 60 seconds for the 200 Http response and it is not finding and it is generating hundreds of alerts. And during a typical outage you do not really want hundreds of alerts for just one server going down.

So this is the reason why throttling of alerts is an also important factor in the alerting system. So this is the basic about alerting. Let’s go to Splunk and let’s look into how Splunk handles this specific aspect. Now, I’m in my Splunk and during the previous videos we had taken a look into the lookup related aspect and we have five items over here. So basically let’s take a use case where what we want is anytime someone like a zero one purchases anything then an alert should be raised. So let’s say that zero one is quite an evil guy and he keeps on purchasing things even though his balance is running out. So what we want is we want to have an alert whenever he purchases something. So let’s look into how we can do that. So we have source type is equal to lookups underscore test. So what we can do is we can go to save as and within save as we can click on alert.

Now, within alert let’s say there are so many aspects over here. However just by having source type is equal to lookup underscore test it does not really tell you much information. So what we want is let’s add one more field, I say zero one and we can say purchase. All right, so there are two fields that we are looking into. One is zero one and second is purchased. So anytime an event occurs within this specific SPL then the alert should be raised. So we’ll go to save as, we’ll click on alert and I’ll say zero one has purchased something. Now alert type is scheduled or real time.

Ideally you want a real time alert and within the triggered conditions you have triggered alert when you have per result when you have number of result so it’s like a simple example for number of result would be Apache. So if number of Http 500 responses are greater than ten within 1 minute, please give an alert. So something similar to that per result is anytime event occurs within this specific query an alert would be raised. Now you even have option for throttling. We already discussed the importance of throttling so if Apache has gone down, you don’t want to receive hundreds of alerts you want to suppress the alerts for that instant of time. So throttling is something that you can do. Now the next important part is the triggered actions. What happens if an event matches this specific SPL? So within the event actions there are various actions that you have first is add to triggered alerts, you have log event, you even have various interesting things like run a script.

So we have already discussed that if you want to run a script on trigger of an alert this is something that you can do send an email and web hook. So web hook is also quite important. For example you have slack in your organization and you want to post a message in slack whenever a zero one has purchased so during those aspects you need to use web books so for our simple example we use add to triggered alerts and we can also specify severity. So I’ll say since zero one is low in balance, the severity is high and I can click on save. So now, if you go into the alerts so here you have the alerts page, you will see that you have one alert. And if you click on this alert, you will basically see that whether there are any fired events which have occurred. So in order to test it out, what we can do is I have a file called as. Lookup. Hyphen sample three. We’ll quickly open this up with item editor and within this you will see that I have a various log files saying that zero one has purchased microphone, zero one has purchased telescope, microscope, et cetera, et cetera.

So what we’ll do is we’ll just copy this source type so that we remember and we’ll go ahead and add a data here. So I’ll click on upload, I’ll select the lookup sample three and we can go ahead and we can upload it. Now within the source type I’ll say lookup underscore test so it will automatically break the events and we’ll leave everything as default here and we’ll click on submit now that we have submitted. Now if you quickly look here you will see that there are a lot of events which have been generated here. So now within the triggered alerts if I quickly do a refresh, you will see that the new trigger have been raised. So these are the alerts which have been raised over here. Now, in case you would have integrated it with email or a web hook, you would have received the alert specific information in email or a specific specific webhooks which you have created accordingly.