54. Logging Module Introduction
I am not saying this because my full-time job is as an instructor at Splunk. It’s not why I’m saying this. I’m telling you, logging is critically important today, and it doesn’t matter if you’re talking about cloud or traditional environments. Let’s talk about some of the advancements in logging technologies today, and let’s talk about how the cloud makes a big difference in this area.
55. Logging
Okay, at the risk of sounding pretty gross here I will never forget when my daughter was growing up and we were teaching her about the potty, one of her favorite books was ‘Everything..’, or ‘Everyone Poops’, I think, or ‘Everything Poops’, or ‘Everybody Poops.’ Something like that, yeah. And guess what? Everyone logs. That’s right, everyone logs. As far as our networking equipment goes, as far as our end user workstations go, everything logs and the cloud can really, very much so be a big benefit in this deluge of logging data that we have. Let’s find out about that now.
So, think about it, you have this on-prem environment and in this on-prem environment you have all of these workstations and you have all of these servers and you have all of these routers and switches and other types of gadgets. They are all generating log files. And these log files are going to dictate the health or they’re going to really depict for us the health and the performance of these systems. So, this is very-very valuable information.
Now think about this. We have all of this information building up, right? These log files are just increasing in their size and the sheer volume and number of them, well, wait a minute here. What if we could call upon the cloud to help us here? What if we could take all of this log information and then intelligently tier it in the storage that we know exists in the cloud and we can storage tier. So, you know, some of the log information that is from zero days to 30 days old, we might keep in a hot type of an access tier, but sure enough, as that data ages, maybe, we are at 128 days for the age of that log information, we can go ahead and have it automatically migrated to a cold storage tier.
Now notice that organizations today, they may have some on-prem type of solution that they have worked out for dealing with all of this logging information that’s constantly done. And they might never have been able to concoct this really elegant design where there’s different tiers of storage for the different age of the logging information. And, of course, there are good old-fashioned hard disk drives (HHD) being used at the cold tier and the new fancy SSDs being used at the hot tier. And remember, there’s automation here where the data is automatically being moved between the tiers.
You see what’s happening? This is one of the reasons that hybrid cloud, by the way, is the most popular cloud model because we can do things on-prem that make sense like have everything log, and then we can call upon the cloud, in the hybrid nature we can call upon the public cloud to help us do something like consume all of this logging information. And of course there’s gonna be all kinds of tools that are available cloud based that would allow you to go in and analyze the logging information.
As you might know, my full-time job is at Splunk where I am a full-time instructor. And at Splunk we do nothing but take your logging data and we bring it into a cloud or non-cloud, you can keep it private if you want with our technologies. And then you’re gonna be analyzing the heck out of that data. You’re gonna be enjoying the views you get of that data and that information based on its cloud storage.
So, yes, logging ties into the cloud today more so than ever. And oh, by the way, just so you’re aware there is one additional thing I should mention to you. You probably know this almost goes without saying. When you have your VMs and you have your containers inside the cloud, of course, they are sending logging information, aren’t they? Yeah, of course. So, in a 100% cloud environment let’s say we’re all cloud all the time, well, logging is still gonna be critically important. And again, we’re gonna be able to call upon mechanisms right within the cloud to help with the logging that is coming from our VMs and our containers in the cloud.
56. Network Management and Monitoring Today
Another of my favorite topics is network management and network monitoring, and how that has changed. Let’s talk about it.
One of the protocols that you need to know about when it comes to network monitoring and network management is the Simple Network Management Protocol (SNMP). This is a tried-and-true method of monitoring your devices. How it works is like this. You have some network device and this could be a client, it could be a router, it could be a switch, it could be a firewall, it could be an IPS device, it could be a multi-layer switch. You get the idea. This is any network device and since Simple Network Management Protocol is an open standard and has been celebrated forever, these networking devices typically have the SNMP management information base (MIB). Yeah, not ‘Men in Black’. No. Management information base. They have this database installed inside them and the software that is required by SNMP to go into the SNMP MIB and retrieve variables. And these variables are populated with, of course, metrics. Common metrics that we would want about the health of the device.
Now, here’s the story of Simple Network Management Protocol. It used to be that we would always joke and say it stands for ‘Security Not My Problem’. Now, it doesn’t really stand for that, right? It doesn’t stand for ‘Security Not My Problem’. But that was our joke about Simple Network Management Protocol, because there’s been three versions that have been accepted and, thank goodness, we are at version three because finally there is security that is featured with the protocol. There was none up until this point. So thank goodness for 2022 and beyond. We are experiencing now version 3 of SNMP and we have for many years by the way. It’s not like 2022 represented the birthplace of the version 3, but notice for the longest time, there was no security. Version 2c uses a community string which is a fancy way to say a password, and this is cleartext information. It is not protected in any way. So, today, we target version 3 of Simple Network Management Protocol. What version three offers, by the way, is various modes of configuration. Yeah, so neat. You can configure it where you’re doing authentication and, maybe, no encryption. So, you don’t care about encrypting the data back and forth. Probably not common, right? Then, of course, you can do authentication and encryption. If you wanted to, you can basically do like no security. So, it’s really wonderful that not only did we finally get security, but we can even have flexible modes of configuring it. So, we can configure it super strong for certain environments and super weak for other environments. So, that is such a great thing.
Well, let’s talk now about something that is a little problematic when it comes to Simple Network Management Protocol. So, here’s how it works. We have this device, any network device, and it has its MIB as we talked about, and we have this SNMP network management device out there. Maybe this is a workstation that we use and it has graphical user interface software on it, and it reaches out and it gets values off of this device that we monitor from this device. Well, think about how SNMP works. It does pulling, so we’d have to like push a button most likely, and we’d have to say, ‘Go fetch the information!’ Notice, we are pulling the information, we’re pulling it down, right? We’re having to go pull it down from the device. Sure, in SNMP, there are things called traps and informs, and these are messages that can be initiated from the thing that’s being monitored, but this is more configuration that we have to do.
So, you probably get a sense for where I’m going here. These days, when it comes to monitoring things, especially things that are in the cloud, we like to use what’s called a telemetry approach. This has really been the new wave in network monitoring. Instead of us having to go and pull down the information we need, telemetry has the device periodically just pushing the information to some location. Oh boy, this is great! Think about all of these gadgets that we have running in the cloud. If they’re all pushing their information to some central collection point, we can then have some software probably also cloud-based that goes in and analyzes all of that information that is being pushed from those devices.
So, we have to be thinking these days more and more, especially when we think about network management inside of the cloud, we need to be thinking more about these push models and this telemetry approach. And think about how easy that is to tune. You can go to these devices and say, ‘Hey, send these certain things every 60 seconds. Send these things every 60 minutes.’ So, you can easily control how much information you’re getting off these devices. And think about it, once you’ve analyzed it, you can easily then in a cloud environment archive the information. So now, it is just being stored maybe for something like six months before it is then dumped. No need to hang onto it, perhaps in your company’s policy after six months.
Well, the cloud sure makes things like network management interesting and I sure hope you found this discussion interesting.
