Request Timeout Bottleneck

Request Timeout Bottleneck

In this tutorial, we are going to discuss about Request Timeout Bottleneck in Istio network. Now Let us understand how the time delay that is getting created by the microservices going to impact other microservices.

For this we have an option to inject the artificial time delay using the fault and fix a delay like that we injected a fixed delay of 2 seconds and the ratings micro service is giving the response. In the previous tutorial we injected a fixed delay of 2 seconds.

Now Let’s go ahead and increase this particular time to 3 seconds and see what’s going to happen. Before that, let’s remember that the default timeout for reviews is 10 seconds and 2.5 seconds for other subset. That means the reviews micro service can wait up to 10 seconds for the version 2 subset and 2.5 seconds for other subset.

Now, as a part of the visual service we are routing all the traffic to version 2. So this particular reviews micro service will be waiting for 10 seconds to get the response from all other microservices that they are calling.

Before discussing further please refer this tutorial for default timeout values. I am using same example here.

Reviews Virtual Serivce

Now, I am going to change the reviews virtual service within the YAML file. What I can do, I can get into the virtual services reviews and get into the yaml tab.

Here It is going to have the destination as version 2. Let me change that to version 3. I have applied the changes. The changes are saved. Let me go ahead and access the webpage. So It should get into the version 3. So it should get me the red color star.

Yes, I am getting the red color star. That means all the traffic is getting into the version 3. Let me access the page few times and get into the graph.

So for reviews I am getting into the version 3. And that is getting into the ratings virtual services and ratings I do have only one subset version 1.

Ratings Virtual Serivce

Now, what I’m going to do I am going to increase the response time of this particular version by changing the fixed delay of the ratings virtual services.

Now Let me get into the ratings virtual service. Let me open the yaml file. Here I’m going to make the fixed delay 2 seconds to 2.8 seconds.

Let me go ahead and access the web UI.

I am not getting any response from the rating service. The reason because we changed the reviews subset to version 1 and version 1 having a timeout as 2.5 seconds. So this reviews 1 will be waiting for 2.5 seconds and beyond that, it will send out a response saying it did not get the response from the ratings and send out the error to the book info and book info will display the error message saying it did not get the response.

So timeout is very very crucial. The reason because within the predefined time duration, I have to give the response to the end-user either an error message or some sort of an information saying I have not received the response and I cannot keep waiting for ever to get the response from the micro services.

Now, let me go ahead and increase this particular duration. That is, I’m going to change the version of reviews to version 2. So that it will have the time out as 10 seconds. Say within virtual services I’m going to get into the reviews, within the YAML file I’ll change the subset to version 2.

So version 2 will have a timeout duration of 10 seconds so it can wait up to 10 seconds. So this time it should get through and it should get the response back.

Yes, I am able to get the response back because the time out for that particular reviews subset is 10 seconds. And having a huge timeout for the microservices is also not acceptable. The reason because the calling microservices may face some issues.

I can prove this by changing the fixed delay of the ratings beyond 3 seconds. The reason because the book info product page is having a timeout of 3 seconds. So it will not wait beyond 3 seconds. And it’s going to give the response back saying it did not get the response, even though the review can wait for 10 seconds. So I am going to change the ratings to delay beyond 3 seconds.

So Let me get into the ratings virtual services as a part of the yaml file.

Here I injected a delay of say 3.8 seconds. And within the reviews virtual service I’m landing in subset 2. So reviews subset 2 that is version 2 can wait up to 10 seconds till the ratings giving the response after 3.8 seconds.

But product page cannot wait beyond 3 seconds. It’s going to give the response back, saying it’s not having the response and give the error message. Let’s go ahead and access the page. Here I should not get the reviews at all.

So it provided the error message saying it have not got the response from the reviews micro service because it is waiting beyond 3 seconds to get the ratings because the ratings we injected, a delay of 3.8 seconds.

So Setting the right time out for each micro service is crucial because within the entire chain, the micro service having the biggest timeout will be acting as a bottleneck within the entire micro service architecture.

Summary

So as a part of this particular tutorial, we discussed about how to change the fixed delay using the fault option available within the virtual services and inject the artificial delay within the microservices and we were making use of the hard coded timeout available as a part of the microservices.

I can over write the timeout using the virtual services as well that we will be seeing it as a part of the next tutorial.