Introduction
I previously wrote an article detailing the retry resilience pattern and its implementation in Polly framework. The article is a bit long and provides an overview of what needs to be done to implement the resilience in an ASP.Net web API setting. This article is more targeted in nature and picks up from the previous article and focuses on implementation of the immediate retry pattern.
Immediate Retry Pattern
Among the retry patterns available in Polly, the immediate retry pattern is perhaps the simplest to understand. The pattern calls the failed service or dependency immediately without any delay between the original and the subsequent retry calls. Since this pattern does not generally provide time to the failed service or dependency to recuperate and serve back, it is generally used where the service or dependency seldom fails. It is also applied where the response times are crucial to maintain a good user experience. Some examples include a heartbeat service, disk write service in an operating system etc.
Figure 1 Retry Pattern Swimlane Diagram
Example
We will simulate a machine-to-machine communication where a Web API project will act as a server and another Web API server will act as a consumer of the service end point exposed by the server project. The consumer project will implement Polly resilience framework. Since I have covered the basics of Polly framework and covered topics like strategies, pipelines and their implementation in a code example in detail in my previous article, I will just gloss over the main details here.
Figure 2 Project structure
A server project serves an action method and randomly allows only 30% of the incoming requests to pass.
[ApiController]
[Route("api/[controller]")]
public class ServiceController : Controller
{
[HttpGet]
[Route("")]
public IActionResult ServiceEndpoint()
{
Random random = new Random();
int dice = random.Next(1, 100);
if (dice < 30)
{
return Ok("Call succeeded. Dice rolled in your favour.");
}
else if (dice > 30 && dice < 50)
{
return StatusCode(StatusCodes.Status502BadGateway, "Bad gateway.");
}
else if(dice > 50 && dice < 70)
{
return StatusCode(StatusCodes.Status500InternalServerError, "Internal server error.");
}
else
{
return StatusCode(StatusCodes.Status408RequestTimeout, "Request Timeout.");
}
}
}
A consumer project consumes server project’s service end point and deploys the Polly resilience project. We create artefacts that will help us implement the immediate retry pattern, which includes, the strategy corresponding to immediate retry pattern, http codes which would spurn the retry logic into action and the predicate that would be invoked in case we receive an unfavourable http status back from the server. The strategy, corresponding options and initialization of the strategy into a pipeline will be done in a special file called “PollyStrategies.cs” placed in a special folder “ResilienceStrategies” folder in the consumer project.
public class PollyStrategies
{
public ResiliencePipelineRegistry<string> StrategyPipelineRegistry { get; private set; }
public ResiliencePipeline<HttpResponseMessage>? ImmediateRetryStrategy { get; private set; }
private RetryStrategyOptions<HttpResponseMessage>? immediateRetryStrategyOptions;
HttpStatusCode[] httpStatusCodesWorthRetrying = new HttpStatusCode[] {
HttpStatusCode.RequestTimeout,// 408
HttpStatusCode.InternalServerError, // 500
HttpStatusCode.BadGateway, // 502
HttpStatusCode.ServiceUnavailable, // 503
HttpStatusCode.GatewayTimeout // 504
};
private void InitializeOptions()
{
immediateRetryStrategyOptions = new RetryStrategyOptions<HttpResponseMessage>()
{
MaxRetryAttempts = 10,
BackoffType = DelayBackoffType.Constant,
Delay = TimeSpan.Zero,
ShouldHandle = new PredicateBuilder<HttpResponseMessage>()
.HandleResult(response => httpStatusCodesWorthRetrying.Contains(response.StatusCode))
.Handle<HttpRequestException>()
.Handle<TimeoutRejectedException>(),
OnRetry = async args => { await Console.Out.WriteLineAsync("ImmediateRetry - Retrying call..."); }
};
}
private void InitializePipelines()
{
ImmediateRetryStrategy = new ResiliencePipelineBuilder<HttpResponseMessage>().AddRetry<HttpResponseMessage>(immediateRetryStrategyOptions).Build();
}
private void RegisterPipelines()
{
StrategyPipelineRegistry = new ResiliencePipelineRegistry<string>();
StrategyPipelineRegistry.TryAddBuilder<HttpResponseMessage>("ImmediateRetry", (builder, context) =>
{
builder.AddPipeline(ImmediateRetryStrategy);
});
}
public PollyStrategies()
{
InitializeOptions();
InitializePipelines();
RegisterPipelines();
}
}
Next, we perform the dependency injection of the PollyStrategies class as a singleton instance. This will make the Polly retry pipeline available across the consumer controllers. Next, we consume the retry pipeline, in our action method as shown in the example below.
Code in Program class in Program.cs file.
public class Program
{
public static void Main(string[] args)
{
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
builder.Services.AddHttpClient();
builder.Services.AddSingleton<PollyStrategies>(new PollyStrategies());
builder.Services.AddControllers();
var app = builder.Build();
// Configure the HTTP request pipeline.
app.UseAuthorization();
app.MapControllers();
app.Run();
}
}
Code in the action method in Consumer controller class.
[ApiController]
[Route("api/[controller]")]
public class ConsumerController : Controller
{
private readonly IHttpClientFactory httpClientFactory;
private readonly PollyStrategies pollyStrategies;
public ConsumerController(IHttpClientFactory _httpclientFactory, PollyStrategies _pollyStrategies)
{
httpClientFactory = _httpclientFactory;
pollyStrategies = _pollyStrategies;
}
public IActionResult ConsumerEndPoint()
{
string url = "http://localhost:5106/api/service";
HttpClient client = httpClientFactory.CreateClient();
HttpResponseMessage response = pollyStrategies
.StrategyPipelineRegistry.GetPipeline<HttpResponseMessage>("ImmediateRetry")
.Execute(() => client.GetAsync(url).Result);
if (response.StatusCode == HttpStatusCode.OK)
{
return Ok("Server responded");
}
else
{
return StatusCode((int)response.StatusCode, "Problem happened with the request") ;
}
}
}
Once we run the project, and execute a request against the consumer project, there’s a fair probability that we will encounter an unfavourable http status code from the server, and our Polly immediate retry pipeline and corresponding strategy will come into action, and will immediate fire back a retry call to the server’s end point, till it gets favorably resolved, subject to the max retries programmed in the strategy.
Figure 3 Immediate Retry Pattern in Action
Figure 4 Desired end result as experienced by a user
Hope this concise introduction to immediate retry pattern was helpful.
References
1. Polly documentation: https://www.pollydocs.org/strategies/retry.html
2. Azure Architecture: https://learn.microsoft.com/en-us/azure/architecture/patterns/retry
3. ThinkMicroServices: http://thinkmicroservices.com/blog/2019/retry-pattern.html
Code
Please follow the link to obtain the code.