F5 101 – Part 3: Maintaining Application Delivery Controller (ADC)

1. Monitoring Overview Part 1

Before we talk about the different ways of monitoring configuration objects let’s first discuss Command Line Interface or CLI. We have two CLI. We have the Advanced Shell or the Linux Batch because this is literally a Linux operating system. This is a Kickstarter where it runs Linux and it loads couple of big IP configuration every time it boots up. We also have the T-Mobile shell or the tmSh. Here in tmSh you have the ability to edit, create and delete configuration objects. Now, later in our configuration demonstration we’re going to compare the difference of the advance shell and tmSh. Now, how do we access Command line interface or CLI? Well, we can also access them via console. We never talk about accessing it via console because we already have a reachable Management IP address. And as we showed you on our previous example, we use SSH.

Now you can use SSH. Both management IP Address and Self IP Address. But via Self IP address we need to verify and configure port lockdown. We also can edit or customize which user can access the advance shell and the tmSh. You may also restrict tmSh and advance shell access by using packet filtering. As I mentioned, VGIP loads configuration during boot up. These are the BigIP Cons, the big IP underscore based Cons and the big DV dot. Now the BigIP cons. This is where your application objects are residing such as virtual servers, pools, snaps, nodes and monitors. The big IP underscore base. Most of these are the network configurations such as Valance Interface, Self IP addresses. Also high availability configurations such as device groups.

We also have the big database up top. These are the system settings such as Hostname. There are also some high availability configurations. Such as high availability settings. tmSh is designed by Hierarchy and you can compare the hierarchical structure in the left pane of our Fib IP GUI. As you can see here, on the left pane you have the modules such as DNS, Local Traffic, Acceleration, Device Management, Network and System.

We can also go here on this module to verify configuration. Or you can create one. As you can see, the hierarchy of the tmSh is first the root. This is the top most. In order for us to view and configure objects you must go to the modules. Now modules examples are network but in tmSh we see it as Net, the system module. Here in tmSh we call it SIS. And the local traffic module in the GUI. In tmSh we call it LTM. Now, under the modules we have submodules or components. Some modules. There are only few of them. The examples are monitor and profiles. You will see a lot of components such as nodes, pools, Virtual, Virtual. In tmSh. We call it their virtual. But in the GUI this is equivalent to the virtual server. Now, there are few tmSh commands and it’s very straightforward. You can easily understand them. We have to create.

We use this command to create an object such as pools, nodes, virtual server, even network objects such as Self, IP, Address and VLANs. We can also use Delete to delete these objects. List is used to verify and view the configuration of these objects. Again, List is to view the configuration and we use Show to view the statistics. So if you want to view the statistics of a pool or a virtual server, you use Show, not List. We can also use save to save our configuration and we use Exit from moving from one hierarchy to another. Let’s say you are in virtual server component and you want to move up to the LTM module. You use exit. Now if you want to go back to the root from LTM, you need to type Exit. Again, we also have Quit, quit is exiting or quitting from the tmSh or TMO shell and you want to go back to the advanced shell. Now, the statistics is how we view the increments of the traffic. We have the bits, we have the pockets and many more. Now, we’ve already used statistics on our previous section. We verify if the pool are load balancing and if the virtual address or the virtual server is receiving requests and traffic. Now, the statistics can be found under the left module.

The first option here Statistics. And under module Statistics we select local traffic to view our application objects such as pools, virtual servers, even I Rules is one of the options under Statistics type. Now here’s an example of viewing statistics for not just pools but every single pool member that is added in this particular pool, as you can see, we have Http underscore pool and we see the traffic of pool numbers one, two and three. Now you can also use statistics to view network information. Under module Statistics we select not local traffic but network. And as you can see, we see here three types of network. Well, four types if you include the management interface, but you have one, one interface, 1213, is unused. We also have management interface. You see the bit counts, the packet, the errors and drops as well.

2. Monitoring Overview Part 2

Now let’s compare the viewing of statistics from a Gui perspective versus the CLI. Now, we already know how to view the statistics of pools and its members, which is under Module Statistics LTM. We select Pools from the Statistics types option. And as you can see, we have two pools. We have the Http and SSH pool, and on each pole we have three pool members. We see the bits, the packets, the connections, the request, and the request queue. We already know this. Now let’s compare it to the CLI view. Okay, now, what we have here is a tmSh or a T Mobile shell access. This is the route. This is our module, and this is the component, which is full. We use show detail. Take note, you can use Show, but this will not provide you enough information to verify the bits, the packets and connection for each pool members. So the detail is optional.

But again, if you want to be more informative, you append with detailed flag or option. Now, I also want to highlight that we went to the component and module. There’s also another way. Instead of going to LTM and pull, and this is the component, by the way component, you also have an option to do something like this show, then LTM and Pull and detail something like this. What this means is you executed the command from the root and you specify the module and the component. And these are the commands. All right, all right, let me just erase this. This is how it would look like if you run the command in the CLI. As you can see here, we have hold on. As you can see, this is our Http pool, and we have the bits in and bits out. Let’s compare 83 and 295 KB. Okay? 83, 295 KB. Okay, all good. Now, if we check the first pool number, the bits in and bits out is thirty two k and one hundred and eight k. So it matches what we have in the GUI.

Our second pool member, we have twenty three k and eighty three k for bits in and bits out. So it matches. Now, I can say that using the GUI in verifying statistics for pools is more convenient and easier from the GUI versus the CLI. It’s just me, but we have some advantages using tmSh or the CLI options, and I’m going to reveal that later. Now, here’s an example of viewing network object statistics from the GUI versus the CLI. From the Statistics module, we select network, and from the statistics type, we select interfaces. And as you can see, we have three interfaces or data interfaces. We have one, one two and one three. We’re not using one three, okay? We’re using one one for our external network, one two for our internal network and for management, let’s compare if the CLI and the GUI matches statistics information. For the one one interface, we have bits in and bits out ten and 18 megabytes. Ten and 18 megabytes. So it’s matching. Okay, we also have 17 and eleven megabytes for the interface. One not 217 and eleven megabytes. So as you can see, this is almost the same. It’s just that one is in GUI, one in CLI. Oh, for the ZLI there is one column here that is not available in the GUI and that is the media. Okay? Now we also have logging in the GUI.

It can be found under the system module and if you hit logs and local traffic as you see this is how it looks like. Now you will see here that virtual server has been added, has been available and we have these pool members it went up. We also have this newly created pool and you’ll see that this pool members went down due to monitoring or help monitoring and later it become available or it went back online also due to help monitoring configuration. Now if we compare the login to CLI is much different. This is not tmSh guys. This is the advanced shelf. This is Linux and we’re using Linux tools such as CD, okay? And tail. We’re going to talk about later what tail is. But remember guys, if you want to view the logging information or the logging message in the advance shell or in the CLI you must be in bar log directory and this file LTM file, this is what you need to view. In this case I use tail and minus ten. This means I want to view the latest or the last ten messages.

Okay? The reason? Because is you might have a very long messages. It can be hundreds or even thousands. So we just want to filter the ten latest messages. Now this is a Linux operating system so you are feel free to use other options such as concatenate or cut or more less. You can also use head if you want to view the first few log messages. We also have the configuration or user configuration set also known as UCS. This is the archive file you’ll be able to view and create your save configuration integrates under system module archive and you will see this page. And in this example we only have one archive file and the name is base underscore Config.

Now you can create a new archive to save configuration, license keys and many more. Now to view the UCS in the CLI you need to go to tmSh and under tmSh you need to go to module sys and UCS. Okay? And as you can see if I hit show it will provide me the details of UCS file. In this case the name is base underscore config UCS and some of the information provided such as the file size, the date it was created, the version of the big IP and many more.

3. Monitoring Demo Part 1

I’m here in my Windows client PC and I’m going to show you how to access our big IP CLI via Management IP address and Self IP address. So I have here Putty, and this is the management IP address of our big IP device. I’m going to create a new one. Hold on, new session. There you go. So I can easily access my big IP device through management IP address via association. Of course. I’m going to click Open and it will ask me for username root with a password of root one. Okay, that is fine. That is very easy because this client PC also has a management connection that it can reach the management interface of our F five big IP device. Now, we already know that this client PC also has an external network that can reach the external interface or external self IP address of our Fib IP. And to verify, I’m going to Open and I’m going to ping ten 131 so I can ping it.

It’s Reachable. Now I am going to attempt to access the CLI via Self IP address 1010, 131. I’m going to click Open now and look, it’s failing. We’re unable to access our F IB IP through the external Self IP address ten dot, ten, dot one, dot 31, even though we know it is reachable. Why is that? Well, because by default, the port lockdown of the self IP address is set to none specifically on the external network. Let me show you what I mean. So under network I click the self IP address and we have the 1010 131. If I click this under the VLANs, we have port lockdown. This is set to none. We are not allowing anything, not SSA, not Https, nothing except for the ICMP packet, by the way. Now, I’m going to custom it or customize it and instead of not allowing any traffic, I’m going to allow port 22, I’m going to click Add, okay? And I’m going to click update.

Now, if we verify the other self IP address such as this 1170, 216, 131, you see that the port lockdown default configuration is allowed default. Now, this is different because this is internal, it’s more secured. Always take note that under external Self IP address, in this case, it’s 1010 dot one, dot 31 and ten dot ten, dot one, dot 33. The default port lockdown settings is allowed none. Now, what I’m going to do is I am going to test if we can now access our big IP device via SSH. Okay? I will go back to my Windows PC and I will simply click restart session. And as you can see, we got the certificate message and we are now about to log in. We got the prompt login as root with a password of root one. There you go. We were able to access our CLI via Self IP address ten dot ten 131 that is mapped to the external network now just to make things clear, I will do the same with the IP address 1010 133. This is the floating self IP address and if I click open it’s still unable to access.

Why? Because we didn’t change the port lockdown value. It’s still set to allow none. But just to verify that we can ping this IP address from the Windows PC. As you can see we got the timeout or network error. This just verifies that we are not allowed to access SSH via this IP address. Now I’m back in my workstation and I’m going to log in again to our CLI. I will use the username root with a password of route one and we are now logged in in our advanced shell. Okay and as I mentioned advanced shell. This is also known as the Linux Bash. The reason why it’s called Linux bash because it literally runs in Linux operating system. If I do you name a it tells us this is a host name vipe one and five trn. com and we’re using kernel version three 10 and this is a Linux operating system with a new Linux license. Now in this advanced shell you can do many different Linux commands and tools such as CDCD. And where are we? We are in the root directory but as you see by default or upon login we logged in in this directory called config.

The reason why it’s called config directory because this is where our configuration files is residing. If I hit LS, you will see many different configuration files but we’ll just focus on the three configuration files we discuss in the slides. The first one is the bigip. com. Now if I concatenate or I can just do more more allows us to view the file or content of the file but page per page. So if I do BigIP con I’ll be able to view the first page. And as you can see these are our objects such as nodes. As you can see we have only three nodes the 170, 216, 21, two, three. We also have the pool Http pool and under Http pool we have three pool members, three nodes listening to port 80 and what else? We also have another pool which is SSH pool. We have two virtual servers, we have the Http vs and the SSH vs. We also have a virtual address, we have the traffic group. What else we have Ihealth. Some of these are default values so we don’t have much object configuration anyway. But later we’re going to create objects and we will verify if this is saved and stored in this file. Now another file name that I want to also show is the big IP base and as you can see this is a different kind of file because you won’t see any objects. You see CRT which is the certificates. You will see device group and you will also see guys. By the way, if you see CM this is equivalent to the device management in our GUI. So you will see all of the traffic group or high availability related configuration under this CM module and the network configuration underneath module is also residing in this file name or this file big IP underscore base. So we have the networks interface 11213.

We have the self IP address take note, self IP address, whether it’s floating or non floating is all residing in the big IP underscore base. Okay? We also have tunnels configuration. We also have some security configurations such as Firewall. This are firewall rule list. It’s not part of our contents but just to let you know that Firewalls policies and rules are added under a siding in this configuration file. Now if I do LS again, it’s not just big IP base and Bigip. com that is residing on your advance shell. There are many more. We also have the license file here. We also have the NTP, we also have the profile base. Now this is the advance shell. Again, there are a lot of things to do in advance shell. I can also verify the interface. We can verify this under tmSh but if you want to view it in a Linux format you may use the ifconfig command. As you can see, I have let me do more. All right, there you go.

So as you can see here we have the external interface and it has an IP address of 1010 131. We also have an internal interface with an IP address of 170 216 131 if you compare this to our tmSh, it has a different view but it’s still the same. It’s just that one is seen by a Linux operating system, one is seen by the tmSh and in tmSh it’s actually self IP address. This is the self IP address mapped to the VLAN name but in Linux it’s not seen as VLAN and self IP address, it’s seen as interface name and the IP address. Okay, now we’re in the advanced shell and as I mentioned, you can use many different Linux tools. This includes TCP dump which we’re going to talk about more in detail later. This allows us to enable packet capture to verify where the traffic goes, what are the IP address, ports and so forth. We also have the big top allows us to monitor in this case nodes, virtual IP and port which is the combination as a virtual server. And as you can see, this is real time.

Now if you want to exit you can just press CTRL C. We’re in the advance shell and again, there are many things to do in advance shell. Now let’s go to tmSh and compare from advance shell you can do tmSh command and here’s what’s going to happen. The prompt will change from this prompt to this prompt. Okay? Big difference though you will see tmSh here or TMOs. We are now in the root of the hierarchy and you cannot do well, still you can do a lot of things. It’s just that if you view the configuration and statistics, it will provide you many information. For example, if I do Show, or if I do list, instead of just viewing one part of the configuration or small part of the configuration, it will provide us 178 items, which is literally everything. Not everything, because some of the configurations are hidden special in the defaults.

We’re going to talk about that in a bit, but what I’m saying is that all of the modules configuration are provided if you do the list command from the root. Now, as I mentioned, tmSh hierarchy is the root and the modules. How to view the modules is very easy. You just hit question mark and it will tell you what are the modules. We have authentication modules. We have CLI. We have CM. Again guys, CM, this is the DM, alright? Device management in the GUI. But here in CLI we call it CM or Centralized Management Configuration. Don’t ask me why it went from DM to CM, I don’t know. Okay? I just want to tell you guys, if you see the device management and we want to view it in our tmSh, it’s under CM. Local traffic in the GUI here is called LTM, which makes sense, okay?

We also have the management, you have the Net. Net also makes more sense because network from the GUI. Here in tmSh, we call it net. Same with the system, we call it Sys utility for utility programs, warm One optimization. This is our acceleration module. Now what I’m going to do is I will use some of the commands such as show, list, create and again, we need to go to a module. I will go to LTM module and here’s what’s going to happen if I do show, it will give me all of the statistics from pools to Vs to snaps. So this is a node statistics, right?

This is still a node statistics, this is a pool statistics, this is a virtual server statistics. And if you want to do some filtering, for example, you just want to view the statistics of a pool or virtual server, all you need to do is go to a specific component. In this case, I will go to virtual and I will just simply do Show. This will only provide us the information of virtual server statistics. This is http underscore PS. This is Sshvs and that’s it. We don’t see pools, we don’t see nodes, okay? Because again, we are in LTM module and virtual server component. We don’t call it virtual server here or Vs, we just call it virtual. Now there’s another way.

If I type exit, I will go back to the LTM module and if I want to view virtual server statistics, only I can do this show virtual. That’s it. Instead of going to the virtual component, I just added the virtual in my command. Show very well and this provides us the same output http underscore PS statistics and SSH underscore PS statistics I also want to show you the exit. Okay. I was in LTM. I went back to the root. If I want to go to another module, I need to go back to the root. TMOs let’s say I want to go to network module under network module I can do show interface and this will give me a statistics or a brief view of our interfaces. 11213 and the management interface.

4. Monitoring Demo Part 2

Now let’s create configuration objects. I’m here in tmSh and when we go to LTM module and under LTM module I want to create a virtual server. I will just go to a virtual server component but the name is virtual and I will use this command create. And I’m going to create a virtual server with the name test underscore Vs. And as you can see, if I hit question mark, it will allows us to know what are the possible option. And there are many options by the way, for test or for virtual server. Excuse me. The minimum will be virtual server name and let’s add the destination. Let’s say 1010 10 and the port. Okay, I will say it’s listening to port 18. There you go. We just created a new virtual server. Now to verify this, all we need to do is type list, command and enter. And since we’re already in the LTM module and virtual component, this will only shows us the virtual server configuration. Let’s do it. Now, if I type list we see Httpbs, we see Sshbs. And as you can see, we have the new virtual server named test underscore Vs and it’s listening to an IP address of 1010 110 and the Http or port 80 is the port. It’s listening. Okay, now we also have an option to just list the specific Vs.

All I need to do is type list and the name is test. We already know that if I type te and hit tab, it will complete the name. Okay, so there’s also an automatic tab completion not only to the built in commands and modules but also to the object name. And if I hit enter, this shows us that our newly configured test can be viewed in our tmSh. Okay, now this is tmSh guys. Okay, here’s the trick. When you configure something in tmSh, the question is, is it also saved in our bigip. com? Think about it, what it is. All right, let’s check it out. Now we’re in tmSh and we want to go back to our advanced shell. Can we do exit? Exit, exit. Oh, it’s not working. No, because if you want to go from tmSh to the advanced shell, exit is not going to work. Exit is only exiting from component to module to the root. That’s what it’s used. The command we use to go back to the advantage shell is quit. We’re now in the advance shell and if we view more bigipip cons and let’s use grip. We want to grip. Vs not working. It doesn’t matter. So this is our configuration.

We have vs it’s seen here. We also have SSH Vs. It’s also seen here in this configuration file. Oops, I miss it. Let’s do it again. All right, so we have SSH and Http Vs and that’s it. This is SSH. This is the Vs for SSH. And as you can see, we associate SSH underscore pool, but that’s it. We don’t have the virtual address that we created 1010 10, I believe. And we don’t have the virtual server test that we currently created. Okay, so why? Why is it not reflecting in our Bigip. com? If I go back to tmSh, and if I do LTM, and guys, by the way, I can actually execute a tmSh command from the advanced shell, okay? I can do that. I can do show and I will do tmSh. No, I will do tmSh show. LTM vertical. I’m not using show. I want to see list. There you go. So from TMSA excuse me, from advance shell, I executed a tmSh command. Okay? I don’t recommend this approach. Why? Because there is no tab completion. It’s better to go to tmSh and from there, execute tmSh commands. Anyway.

So I got the list, the configuration. We have SSH, we have the test, and we have Http, but it didn’t reflect in our big IP configuration file. Why is that? Well, because it is not yet saved. When you use CLI and do some configuration or deletion, it will not reflect in the big IP. Meaning, if you reboot the big IP device, your configuration will be gone or that new leak new configuration will be gone. So I need to go to tmSh and I need to use this command save this configuration. This allows us to save the configuration from PMSH to BigIP conf, big IP base, and big IP userconf. And if we go back to our advanced shell and do more pip all right, let’s verify Http Vs. It’s here. Sash’s it’s here. And there you go. Test underscore virtual server is now added in our bagpipe conf.

Okay, yeah, we didn’t use 100 plus. We used 1010 10 only soliciting to port 80. We didn’t associate pulls or any other configuration because it’s just used for testing. Now, what I’m going to do is I will verify if this exists in my GUI. Okay, so we got logged out. Let’s log back in. All right, if I go to local traffic and sorry, local Traffic virtual server. There you go. As you see, we have our test underscore Vs, and it’s now available in the GUI as well. Now we go back to tmSh, and we want to delete this because we’re not going to use this virtual server anyway in our next examples. So I will go back to LTM virtual and I’m going to delete test and I hit the tab, it automatically completes this virtual server name. I’m going to delete it, and if I do list, it’s gone.

We only have Http and SSH virtual server. And here’s what I’m going to do. I’m going to save it. There we go. Okay, so I just saved our configuration, and it should delete our test underscore PS. Okay, now, before we go back to the GUI, I just want to show you how to view the log messages. You need to go from the advanced shell I just type quit and enter. Now we’re back in the advanced shell. I need to go to VAR log. And if I do LS you will see many files. We’re not very concerned about the files. What we’re concerned about is just one file. The LTM by the LTM or concatenate. This will gives us the login message for the LTM. And as you can see, this is what we only see. We don’t have much, but this is how to view the log messages. Now let’s go back to the GUI.