14. HTTP Troubleshooting Part 1
Now let’s move to application connection networks. What I have here is a server, but this time it’s not listening to port 80, it’s listening to port four four three. And it is added to a pool named S underscore pool one. It’s only one pool number and this pool is associated to a virtual server Https underscore Vs with an IP address of ten 10110 listening on port four four three. Now, our issue is the client is unable to access the web application and you as an administrator, you check all of the application objects, the nodes, the pool, the pool member. We only have one, the virtual server. All of these objects are available green circle.
Now in the exam they may provide you a net stat output. If you see something like this, this is from the server side. It shows us that port at and port 443 are both running. If you see this, this means that this is not an application issue. It’s probably a network issue, just like from our previous example where our servers route entry is incorrect.But if you see this kind of output where we only have a port 80 that is listening, that is currently running, and the issue we have here is web applications. It’s not specific to Http or Https, it’s just say Web applications. It’s probably the cause of the issue is related to server.
The server is supposed to list in on port four four three, but on the Netstat output, four four three is not running, but only port 80. Well, port 80 is not used by the virtual server. It is also not the application your client is using. Well, it didn’t specify here, but again, this is mostly the correct one compared to the other options. Again, you have to do a lot of analysis and comparison amongst all of the options and of course the output in this case, since we only have a port 80 listening from this Netstat. The problem may be related to server listening to port four four three because in our Netstat output it is not running.
15. HTTP Troubleshooting Part 2
In our second example, we have three servers behind our big IP appliance. And these three servers are listening to port 80. And they are added in a pool called Http underscore pool. This pool is associated to our Http virtual server with an IP address of 1010, dot one 10, listening to port 80. Now, the client is trying to access this Web application, but is always unaccessible. Now, you, as an Fib IP specialist, log into your big IP GUI and verify the objects such as nodes, pool members, pool, and even virtual servers are marked as available or green circle. Now, it is your task to troubleshoot what’s the cause of this issue. Again, the client cannot access the Web application.
And anytime he requests traffic to ten dot ten, dot one, dot 100, he’s getting an Http editor. We’re here in our big ipadvant show. Now let’s run the command TCP dump, and I’m going to specify our internal Vlan. Now, if I run this command, we see that there are a bunch of Icmp requests and reply, maybe because this is the help monitor used by our Notes or to our Notes. Now, I’m going to add a filter port 80 because we want to figure out the traffic and analyze the traffic coming from the big IP down to our servers. And I’m not going to specify a server IP address because the big IP may do the load balancing to all of these servers. So here’s what I’m going to do. I am going to hit enter now.
And as you see, there’s no result yet, maybe because there’s still no traffic. Next, I’m going to log into our Windows client and initiate the Http traffic using our Web browser. I’m going to hit refresh. And look at that. We are now seeing our packet coming from the big IP to one or multiple servers. As you see here, this is the traffic coming from the big IP, right? And it is initiating a TCP three way handshake to the specific pool member. Okay, 172 dot 1621. And the flag, or the TCP flag is TCPC. Now, the server responded this is the server responded back to the big IP. Same IP address, same port with a flagr, meaning reset flag. And the big IP finalize the reset flag by sending back okay, you’re not responding.
You send me a reset flag. I am going to finalize this TCP failed reset TCP three way hash by sending another TCP flag to that destination server. And this will go continuously not only to the server one, but also to the second pool member. Initiate TCP scene. And it finalized the TCP with a flag of our or reset. So we already know that our servers are not running Http services. What I’m going to do is I will go back to our big IP GUI, and from there we will check what’s the help monitor’s current configuration. All right, so we know that all of these application objects are in green circle or marked as available. Let’s verify the help monitors. Why are the pool members it’s mark green circle.
So first let’s go to the nodes and let’s verify the default monitor. And as you can see, it’s using Icmp. We verify this by using TCP dump and we specify the internal Vlan. And we saw the Icmp packets coming from the big IP to three pool members, or in that case, only a node, because we are only seeing a destination IP address. Now let’s go to the pools and we verify what is the configuration of our Http pool. And look at that. It’s not using Http held monitors. Instead, it’s using a gateway Icmp, which is a network based icmp based health monitors. And this is the reason why old pool members is marked online, because old pool members are responding to the Icmp echo request.
So what I will do next is I’m going to remove this gateway Icmp help monitor and replace it with Http based help monitors. Now upon clicking Update, it will start sending Http requests to the server. And if the server respond, it will mark the server green circle. But this is not what we’re expecting, because in our TCP Dom, we are seeing that the servers are not responding back with the TCPC. So I’m going to click update now. Okay, we’ll wait for a few seconds and we’ll click the network map to view the summary of the health status of all of the pool members and the pool and vertical servers. I’m going to click Network Map now and let’s see what’s going to happen.
There you go. Just like what we expected. All pool members are marked offline or red diamond. Why? Because Http is not running to all three pool members. Okay, so what happened is all three pool members are offline. Since all three pool members are offline, http pool will become offline as well because there is no available pool members. And the Http vs will just inherit the status from the Http pool. If I hover my mouse to all three pool members, we see that the nodes are marked as available or online. Why? Because all three nodes are responding to the Icmp request, which is the help monitor configured for the node default.
Again, the issue is all three poll members are not running Http services, and that’s why our clients unable to access the web application. We logged into our big IP GUI, and we confirmed that objects such as nodes, pool members, pool, and virtual servers are marked as online or green circle. We also logged into our big Ibcli, and from our advanced shell we run a command called TCP dump, TCP dump and we filter with a port 80 or our Http application. We see that the big IP is trying to initiate a TCP scene, but these pool members is not responding. So the big IP will then conclude that. Hey, since you are not responding with my TCP scene, I will just send you a TCP reset flag.
And reset flag allows us to confirm that these servers are not running Http services or in short, Http port is closed. It’s not open. And what we did back in our GUI is that we found out under pools health monitors configuration, it is assigned using the network base or the Icmp based health monitors. And what we did next is we remove that Icmp based help monitor and replace it with Http base since we know that the servers are listening or supposed to listen with Http or TCP port at application. Now, what happened after we associate that Http health monitor, these three servers are marked offline or red diamond. Why is that? Well, because this three servers doesn’t have Http application running.
16. HTTP Troubleshooting Part 3
In our third example, it’s a bit different because this time the web application is working and we have three servers all listening to port 80. It is added to a pool that is associated with the virtual server. This client assistance traffic to our virtual server. The big IP process it will do the load balancing and sends it to the pool members. Everything is normal. But the issue here is that since the big IP is receiving frequent requests to a site that contains a large amount of static content, these are images, CSS files and Javascripts. As an F five administrators needs to enable optimization because number of clients continuously increasing. Now, what feature needs to be enabled for you to optimize this large number of requests from multiple clients? When we enable Http caching, it works a little different.
It’s something like this clients send Http request to the virtual server. The big IP will process it to the load balancing and select a pool member. As the pool member received a request, it will respond back to the big IP with all of these contents. The big IP will store these contents in its system memory, this content such as images, CSS and JavaScript files. Now it works the same. As the big IP selected another pool, it will respond back to the big IP with all of its contents. The big IP again stores this content to its memory and sends it back to the client. Now, assuming we have other clients requesting Http traffic to the same virtual server, instead of the big IP will get all of these contents from all these three servers.
This is what’s going to happen. Client sends Http requests again to the same virtual server and the big IP will say a some of the contents you need, I already have it. So I’m not going to get all of those files such as Images, CSS and JavaScript to every single servers, but instead I’m going to send it back to you directly. Okay? And again, the goal of Http caching is to send some of the contents from the big IP and not from the servers. Maybe some of you thought that Http compression is what we need to enable in order to optimize Http traffic from the client to the VGIP device. But here let’s compare what’s the difference between Http caching? But in our GUI it’s called web acceleration versus Http compression, also known as content encoding.
First, Http caching is a collection of Http objects stored in the big IP system. Memory and subsequent connections can reduce to reduce traffic load from the servers.So again, we’re helping the servers to send some of the files, some of the content back to the client. And it’s actually done that by the VIP device http. Caching. The goal is to reduce the need to send frequent requests for the same object. Again, these are the images, CSS files, JavaScript and eliminate the need to send all responses from the servers. Again, it’s the big IP can send the response back to the client. And here are some of the content the caching can store and send back to the client.
These are 200 203, three 1401 Http responses as well as images, JavaScript and CSS files. And this is actually matching the definition of our problem in the previous slide. Now for the Http compression it works differently because what it compressed is the body. And when I say body, this can be HTML files, not only images, it can also be document files. Again, not images and CSS such as PDF and media files as well video and audio. Now for Http compression it works differently. First the client sends Http request, the big IP reads it and it removes the accept encoding. So it will remove the accept encoding from the header and pass this to the server.
The reason why the big IP do this is to force the server not to do the compression. Once the server respond back to the big IP, the big IP inserts the content encoding. Content encoding is also known as the Http compression and it’s doing the compression based on what’s available from the web browser. It can be either gun, zip or deflate. These are the two most common Http compression method and send the compressed data back to the client. Now that is the real goal of Http compression is to reduce the size of the Http request body using available method. Again, the available methods are gun, zip and deflate from client to big IP back to the client.
So if you’re going to compare Http compression and Caching, it’s a bit different because caching stores these objects, this contents from the big IP memory and the big IP is the one who will respond back to the client. And this only works in limited files such as images and CSS. JavaScript is not recommended to cache all available objects, while compression reduce the size of Http data. So again, in our example or this exercise it’s Http caching needed to be enabled for optimization. And don’t forget, if you want to enable either Http compression or Http caching, you must create a profile. And for you to use this profile and associate to the virtual server, make sure Http profile is enabled because acceleration profile is dependent to Http gtp profile.
17. HTTP Troubleshooting Part 4
Before we demonstrate the different types of Http status codes, let’s have a little refresher. Http status code is a three digit integer seen in Http response header and it identifies the general category of response on the first digit. The example of an Http status code is 200 okay and it is next to the Http version in again Http response header. This is the version and 200 okay means the Http request initiated by the client is successful. We also have Http 404 not found and this means that there is unsuccessful response back to the client. Now we have five general category of responses. The first one is the 100 level and this indicates an informational message only.
We also have 200, indicates a success or some level of success. We also have the 300 and this allows our server to redirect the client to another URL. So if there is a redirection happening, it’s under the 300 level. We also have the 400 level and this indicates an error on the client side. And lastly we have the 500 level indicates an error on the server side. Our first demo is Http 302 redirect. What will happen is as the client sends Http requests, the big IP will redirect to another URL. How will we do this? We’re going to use an I will redirect and associate this to our virtual server. I’m here in our Windows client PC and we’re going to test Http underscore Vs. I’m going to type 1010 100 and it is responding.
If I click refresh, it load balance to server one, refresh again, refresh again. It loads balance to server three. Now server two. And this confirms that load balancing is working. Now I’m going to minimize and configure our Http underscore Vs. This time we’re going to assign an I rule. What’s the I rule for? Well, we want to redirect the traffic from this virtual server Http underscore vs and redirect it to Https underscore Vs with an IP address of ten 10110. Listening on port 443 I’m going to click Resources. And below here there is an idol configuration. I’m going to click Manage and from the available script I will just go down the last I will script with the name of Sys. Https.
Redirect is what we’re going to enable. I just click the chip run sign and hit finish. Oh, it’s not working. Or it is not successfully enabled. Why is that? Because it requires to enable Http profile. So I will go back to the Properties page and from here I will enable Http profile by selecting Http next to Http profile configuration I’m going to click Update. Now our Http profile is enabled. We can go back to our idle configuration and reassign the reader Rex trip. And this time I’m pretty sure it will be successful. Yes, it is now successful associated to our Vs. Now let’s test if our redirect will work. I’m back in our Windows client PC and I will hit refresh.
Currently we are or we are connected to Http application with the blue color on the page. I’m going to hit refresh again. For those who don’t remember, blue collar means Http only. And if I verify here, it says the connection side is not secure. And this is http I’m going to hit refresh. It’s still unsecured, but this is Https because we see certificates and you see the page is now using a color red, meaning it’s Https. And what we’re going to do is under the Developers tool, we want to verify the status code. Okay, I will close again this browser. We just verify that the redirection is working. And here’s what I’m going to do. I will go to Monitor tools, developer tools, and I’m going to type 1010, 100. Again.
This is http and to make sure, I will add Http and then I will hit Enter. You will see the Http status code. Not only the status code, the Http header. Both requests and response will be viewed here. Okay, I’m going to click enter now. And there you go. Now, the first entry here, 1010 100. As you see, the status quo is 302 because it redirect from Http IP address 1010 100 listed on port 18 to Https. Same IP address, but different port. This time it’s four, four three. Let’s verify. So as you see, this is Http 1010 100. This ad code is 302 found. And we have the request setter. We have the response header. This is pretty normal. But if you look at the next entry, this is this page, the page with a red background.
And the protocol that we’re using is Https, same IP address, which is 1010 one 10. And look at the status quo now it’s 200. Okay? Because what happened is the Http destination 1010 100 for 80 redirects to Https 1010 100, port four, four three. It’s just a redirection script and it’s pretty straightforward. All of the other objects are part of the Https. If I click the F five logo, it is Https, the IP address image and the image name, as well as other objects such as let’s hit the global CSS. It’s still under the Https under and the path is slash CSS global dot CSS still status code 200. Okay, so in short, the only three two is D spage, the ten dot, ten dot, one dot 100 listening on port 18.
18. HTTP Troubleshooting Part 5
Now we’re going to continue testing the different types of Http status code. But this time we’re going to use the Internet. So I am going to connect to Zurion. com via the Internet and test this three Http status editor code. The first one will be Http found. Next is Http 401 unauthorized and both of these are 400 level. Now the 400 level indicates that there is an error on the client side. And lastly we’re going to demonstrate Http 502 bad gateway and this is on the 500 level status code, which means that it’s an indication of an error on the server side. First thing we will do is to access a site or a URL that doesn’t exist. So I’m here in my web browser, I’m going to type Wwrin. com and I’m going to add a Uri again that doesn’t exist.
I will add My Beer and as you can see we got a 404 error not found. Okay, I will now go to my more tools, developer Tools to dissect the Http request and Http response of this specific page. Okay, I will click Developers Tools now and as you can see it has already selected the network tab. I select the filter all. As you can see there’s no entry yet. What I’m going to do next is I’m going to hit refresh again. So it will add entries and again we’re going to dissect the Http messages. I’ll hit refresh now and as you can see, we don’t only have one items, we also have few and some of these are images. Now first thing I will show you is the My beer page. And as you can see we get the header 404.
And if I show you the response header, it shows you the server is enginex content type, text, HTML. And unlike what we did in our previous example, it didn’t show us the type of application, but that’s okay. Now also take note that it’s not only the page that uses content, there’s already also attached images. And the reason why is because this is not your generic Http code 404. Some of these are customized or not some, this page is 100% customized such as images. So what you see here, the 404 here, it is an image as well as the top. This is top PNG as you can see. And this part here, this is also an image. It’s called the bottom PNG. We also have JavaScript and CSS files. So again, if I click these other contents, there are 200.
OK, so the only page that is responding with Http code 404 is this my beer URL. Next is we’re going to visit a specific URL that will require us authentication. So I will just edit our URL zurion. com off, I’ll hit enter and as you can see it is requiring us a username and password. I will add let’s say Zurion, the password of F five. No, we’re not getting any authentication or should I say a login, successful authentication. But what we’re going to do is I will click again Developers Tool and I’m going to click Network. As you can see there’s still no entry. But if I continue adding or trying to be authenticated, you will see there’s another entry. Now to provide you more and complete information, I’m just going to cancel it.
We will get the same result anyway going to cancel. As you can see on the main page, we are getting 401 authorization required. And if I look at my developer tools under Network, there’s an entry here Auth. And as you can see the status code is 401 and authorized. And this is pretty straightforward. The reason why we got this, because we were not successfully provide the username and password, we’re not authenticated as well as unauthorized. And here is our last example. I will change the URL to zurich. com 502 and as you can see it returned an Http code 502 bad gateway. Now we will go back again to more tools, developer tools to see the details of the Http messages.
I’m going to hit refresh and as you can see we get an entry and the request methods get the status quo that we are getting as what we are seeing in the main page is 502 bad gateway. If I click Source I see again the server Nginx and that’s it. And the reason why we get this 502 error and what we talk about is the 500 level is the server side errors, right? And what we did is we configured a container and the container that we created is a proxy or a gateway that we intentionally receive an invalid response from an upstream server. Now, compared to the 400 errors, the error is based on the client side. The first example, the 404 not found the reason why we were getting that error because our URL is not correct, it doesn’t exist. And the second one is the 401 unauthorized because we are not able to provide upcoming correct username and password.
