Quality of Service

Label: QoS

Ocelot currently supports a single QoS capability. It allows you to configure, on a per-route basis, the use of a circuit breaker when making requests to downstream services. This feature leverages a superb .NET library known as Polly. For more information, visit their official repository.

Installation

To use the Administration API, the first step is to import the relevant NuGet package:

Install-Package Ocelot.Provider.Polly

Next, within your ConfigureServices method, to incorporate Polly services, invoke the AddPolly() extension on the OcelotBuilder returned by AddOcelot() [1] as shown below:

services.AddOcelot()
    .AddPolly();

Configuration

Then add the following section to a Route configuration:

"QoSOptions": {
  "ExceptionsAllowedBeforeBreaking": 3,
  "DurationOfBreak": 1000,
  "TimeoutValue": 5000
}

  • You must set a number equal or greater than 2 against ExceptionsAllowedBeforeBreaking for this rule to be implemented. [2]

  • DurationOfBreak means the circuit breaker will stay open for 1 second after it is tripped.

  • TimeoutValue means if a request takes more than 5 seconds, it will automatically be timed out.

Circuit Breaker strategy

The options ExceptionsAllowedBeforeBreaking and DurationOfBreak can be configured independently of TimeoutValue:

"QoSOptions": {
  "ExceptionsAllowedBeforeBreaking": 3,
  "DurationOfBreak": 1000
}

Alternatively, you may omit DurationOfBreak to default to the implicit 5 seconds as per Polly documentation:

"QoSOptions": {
  "ExceptionsAllowedBeforeBreaking": 3
}

This setup activates only the Circuit breaker strategy.

Timeout strategy

The TimeoutValue can be configured independently from the ExceptionsAllowedBeforeBreaking and DurationOfBreak settings:

"QoSOptions": {
  "TimeoutValue": 5000
}

This setup activates only the Timeout strategy.

Notes

  1. Without a QoS section, QoS will not be utilized, and Ocelot will impose a default timeout of 90 seconds for all downstream requests. To request configurability, please open an issue. [2]

  2. Polly V7 syntax is no longer supported as of version 23.2. [3]

  3. For Polly version 8 and above, the following constraints on values are specified in the documentation:

    • The ExceptionsAllowedBeforeBreaking value must be 2 or higher.

    • The DurationOfBreak value must exceed 500 milliseconds, defaulting to 5000 milliseconds (5 seconds) if unspecified or if the value is 500 milliseconds or less.

    • The TimeoutValue must be over 10 milliseconds.

    Consult the Resilience strategies documentation for a detailed understanding of each option.

Extensibility [3]

If you want to use your ResiliencePipeline<T> provider, you can use the following syntax:

 services.AddOcelot()
     .AddPolly<MyProvider>();
// MyProvider should implement IPollyQoSResiliencePipelineProvider<HttpResponseMessage>
// Note: you can use standard provider PollyQoSResiliencePipelineProvider

If, in addition, you want to use your own DelegatingHandler, you can use the following syntax:

 services.AddOcelot()
     .AddPolly<MyProvider>(MyQosDelegatingHandlerDelegate);
// MyProvider should implement IPollyQoSResiliencePipelineProvider<HttpResponseMessage>
// Note: you can use standard provider PollyQoSResiliencePipelineProvider
// MyQosDelegatingHandlerDelegate is a delegate use to get a DelegatingHandler

And finally, if you want to define your own set of exceptions to map, you can use the following syntax:

services.AddOcelot()
    .AddPolly<MyProvider>(MyErrorMapping);
// MyProvider should implement IPollyQoSResiliencePipelineProvider<HttpResponseMessage>
// Note: you can use standard provider PollyQoSResiliencePipelineProvider

// MyErrorMapping is a Dictionary<Type, Func<Exception, Error>>, eg:
private static readonly Dictionary<Type, Func<Exception, Error>> MyErrorMapping = new()
{
    {typeof(TaskCanceledException), CreateError},
    {typeof(TimeoutRejectedException), CreateError},
    {typeof(BrokenCircuitException), CreateError},
    {typeof(BrokenCircuitException<HttpResponseMessage>), CreateError},
};
private static Error CreateError(Exception e) => new RequestTimedOutError(e);