I’m going to describe two features of Spring Cloud Gateway: retrying based on GatewayFilter pattern and timeouts based on a global configuration. In some previous articles in this series I have described rate limiting based on Redis, and a circuit breaker pattern built with Resilience4J. For more details about those two features.
IMPLEMENTATION
AND TESTING
As
you probably know most of the operations in Spring Cloud Gateway are realized
using filter pattern, which is an implementation of Spring Framework
GatewayFilter. Here, we can modify incoming requests and outgoing responses
before or after sending the downstream request.
The same as for examples described in my two previous articles about
Spring Cloud Gateway we will build JUnit test class. It leverages
Testcontainers MockServer for running mock exposing REST endpoints.
Before running
the test we need to prepare a sample route containing Retry filter. When
defining this type of GatewayFilter we may set multiple parameters. Typically
you will use the following three of them:
retries
– the number of retries that should be attempted for a single incoming request.
The default value of this property is 3
statuses – the list of HTTP status codes that should be retried,
represented by using org.springframework.http.HttpStatus enum name.
backoff – the policy used for calculating timeouts between
subsequent retry attempts. By default this property is disabled.
Let’s start from the simplest scenario – using default values of
parameters. In that case we just need to set a name of GatewayFilter for a
route – Retry.
@ClassRule
public static
MockServerContainer mockServer = new MockServerContainer();
@BeforeClass
public
static void init() {
System.setProperty("spring.cloud.gateway.routes[0].id",
"account-service");
System.setProperty("spring.cloud.gateway.routes[0].uri",
"http://192.168.99.100:"
+ mockServer.getServerPort());
System.setProperty("spring.cloud.gateway.routes[0].predicates[0]",
"Path=/account/**");
System.setProperty("spring.cloud.gateway.routes[0].filters[0]",
"RewritePath=/account/(?<path>.*), /$\\{path}");
System.setProperty("spring.cloud.gateway.routes[0].filters[1].name",
"Retry");
MockServerClient
client = new MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort());
client.when(HttpRequest.request()
.withPath("/1"),
Times.exactly(3))
.respond(response()
.withStatusCode(500)
.withBody("{\"errorCode\" 
"5.01\"}")
.withHeader("Content-Type",
"application/json")); client.when(HttpRequest.request()
.withPath("/1"))
.respond(response()
.withBody("{\"id\":1,\"number\" "1234567891\"}")
.withHeader("Content-Type",
"application/json"));
// OTHER RULES
}
Our
test method is very simple. It is just using Spring Framework TestRestTemplate to perform a single call to the test endpoint.
@Autowired
TestRestTemplate template;
@Test
public
void testAccountService() { LOGGER.info("Sending /1...");
ResponseEntity r = template.exchange("/account/{id}",
HttpMethod.GET, null, Account.class, 1); LOGGER.info("Received:
status->{}, payload->{}", r.getStatusCodeValue(), r.getBody());
Assert.assertEquals(200, r.getStatusCodeValue());
}
Before
running the test we will change a logging level for Spring Cloud Gateway logs,
to see the additional information about the retrying process.
logging.level.org.springframework.cloud.gateway.filter.factory:
TRACE
Since
we didn’t set any backoff policy the subsequent attempts were replied without
any delay. As you see on the picture below, a default number of retries is 3,
and the filter is trying to retry all HTTP 5XX codes (SERVER_ERROR).
Now,
let’s provide a little more advanced configuration. We can change the number of
retries and set the exact HTTP status code for retrying instead of the series
of codes. In our case a retried status code is HTTP 500, since it is returned
by our mock endpoint. We can also enable backoff retrying policy beginning from
50ms to max 500ms. The factor is 2 what means that the backoff is calculated by
using formula prevBackoff * factor. A formula is becoming slightly different
when you set property basedOnPreviousValue to false – firstBackoff * (factor ^
n). Here’s the appropriate configuration for our current test.
@ClassRule
public static
MockServerContainer mockServer = new MockServerContainer();
@BeforeClass
public
static void init() { System.setProperty("spring.cloud.gateway.routes[0].id",
"account-service");
System.setProperty("spring.cloud.gateway.routes[0].uri",
"http://192.168.99.100:"
+ mockServer.getServerPort());
System.setProperty("spring.cloud.gateway.routes[0].predicates[0]",
"Path=/account/**");
System.setProperty("spring.cloud.gateway.routes[0].filters[0]",
"RewritePath=/account/(?<path>.*), /$\\{path}");
System.setProperty("spring.cloud.gateway.routes[0].filters[1].name",
"Retry");
System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.retries",
"10");
System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.statuses",
"INTERNAL_SERVER_ERROR"); System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.firstBackoff",
"50ms");
System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.maxBackoff",
"500ms"); System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.factor",
"2");
System.setProperty("spring.cloud.gateway.routes[0].filters[1].args.backoff.basedOnPreviousValue",
"true"); MockServerClient client = new
MockServerClient(mockServer.getContainerIpAddress(), mockServer.getServerPort());
client.when(HttpRequest.request()
.withPath("/1"),
Times.exactly(3))
.respond(response()
.withStatusCode(500)
.withBody("{\"errorCode\" "5.01\"}")
.withHeader("Content-Type",
"application/json")); client.when(HttpRequest.request()
.withPath("/1"))
.respond(response()
.withBody("{\"id\":1,\"number\" "1234567891\"}")
.withHeader("Content-Type",
"application/json"));
// OTHER RULES
}
If
you run the same test one more time with a new configuration the logs look a
little different. I have highlighted the most important differences in the
picture below. As you see the current number of retries 10 only for HTTP 500
status. After setting a backoff policy the first retry attempt is performed
after 50ms, the second after 100ms, the third after 200ms etc.
We
have already analysed the retry mechanism in Spring Cloud Gateway. Timeouts is
another important aspect of request routing. With Spring Cloud Gateway we may
easily set a global read and connect timeout.
Alternatively we may also define them for each route separately.
Let’s add the following property to our test route definition. It sets a global
timeout on 100ms. Now, our test route contains a test Retry filter with newly
adres global read timeout on 100ms.
System.setProperty("spring.cloud.gateway.httpclient.response-timeout",
"100ms");
Alternatively,
we may set timeout per single route. If we would prefer such a solution here a
line we should add to our sample test.
System.setProperty("spring.cloud.gateway.routes[1].metadata.response-timeout",
"100");
Then
we define another test endpoint available under context path /2 with 200ms
delay. Our current test method is pretty similar to the previous one, except
that we are expecting HTTP 504 as a result.
@Test
public
void testAccountServiceFail() { LOGGER.info("Sending /2...");
ResponseEntity<Account> r =
template.exchange("/account/{id}", HttpMethod.GET, null,
Account.class, 2); LOGGER.info("Received: status->{},
payload->{}", r.getStatusCodeValue(), r.getBody());
Assert.assertEquals(504, r.getStatusCodeValue());
}
3 Comments
nice pratap
ReplyDeleteVery nice really amazing post thank for this post pratap
ReplyDeleteAwesome Job Pratap
ReplyDelete